prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  Improve the `Re-use the best values` tutorial page
  ### What is an issue?
  
  There are some minor issues on the Re-use the best values tutorial page.
  
  ## Description
  - [x] On the [Re-use the best values](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/010_reuse_best_trial.html) page, We should update the title as `re-use the best parameters` instead of `re-use the best values`. As the best value means the best objective value in most of the Optuna references.
  - [ ] Mention `Study.best_trials` on the page for multi-objective optimization.
  
  
  Context: https://github.com/optuna/optuna/pull/3396#pullrequestreview-917028874
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `tutorial/20_recipes/010_reuse_best_trial.py`
  Content:
  ```
  1 """
  2 .. _reuse_best_trial:
  3 
  4 Re-use the best values
  5 ==========================
  6 
  7 In some cases, you may want to re-evaluate the objective function with the best
  8 hyperparameters again after the hyperparameter optimization.
  9 
  10 For example,
  11 
  12 - You have found good hyperparameters with Optuna and want to run a similar `objective` function using the best hyperparameters found so far to further analyze the results, or
  13 - You have optimized with Optuna using a partial dataset to reduce training time. After the hyperparameter tuning, you want to train the model using the whole dataset with the best hyperparameter values found.
  14 
  15 :class:`~optuna.study.Study.best_trial` provides an interface to re-evaluate the objective function with the current best hyperparameter values.
  16 
  17 This tutorial shows an example of how to re-run a different `objective` function with the current best values, like the first example above.
  18 
  19 
  20 Investigating the best model further
  21 -------------------------------------
  22 
  23 Let's consider a classical supervised classification problem with Optuna as follows:
  24 """
  25 
  26 from sklearn import metrics
  27 from sklearn.datasets import make_classification
  28 from sklearn.linear_model import LogisticRegression
  29 from sklearn.model_selection import train_test_split
  30 
  31 
  32 import optuna
  33 
  34 
  35 def objective(trial):
  36     X, y = make_classification(n_features=10, random_state=1)
  37     X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)
  38 
  39     C = trial.suggest_float("C", 1e-7, 10.0, log=True)
  40 
  41     clf = LogisticRegression(C=C)
  42     clf.fit(X_train, y_train)
  43 
  44     return clf.score(X_test, y_test)
  45 
  46 
  47 study = optuna.create_study(direction="maximize")
  48 study.optimize(objective, n_trials=10)
  49 
  50 print(study.best_trial.value)  # Show the best value.
  51 
  52 ###################################################################################################
  53 # Suppose after the hyperparameter optimization, you want to calculate other evaluation metrics
  54 # such as recall, precision, and f1-score on the same dataset.
  55 # You can define another objective function that shares most of the ``objective``
  56 # function to reproduce the model with the best hyperparameters.
  57 
  58 
  59 def detailed_objective(trial):
  60     # Use same code objective to reproduce the best model
  61     X, y = make_classification(n_features=10, random_state=1)
  62     X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)
  63 
  64     C = trial.suggest_float("C", 1e-7, 10.0, log=True)
  65 
  66     clf = LogisticRegression(C=C)
  67     clf.fit(X_train, y_train)
  68 
  69     # calculate more evaluation metrics
  70     pred = clf.predict(X_test)
  71 
  72     acc = metrics.accuracy_score(pred, y_test)
  73     recall = metrics.recall_score(pred, y_test)
  74     precision = metrics.precision_score(pred, y_test)
  75     f1 = metrics.f1_score(pred, y_test)
  76 
  77     return acc, f1, recall, precision
  78 
  79 
  80 ###################################################################################################
  81 # Pass ``study.best_trial`` as the argument of ``detailed_objective``.
  82 
  83 detailed_objective(study.best_trial)  # calculate acc, f1, recall, and precision
  84 
  85 ###################################################################################################
  86 # The difference between :class:`~optuna.study.Study.best_trial` and ordinal trials
  87 # ----------------------------------------------------------------------------------
  88 #
  89 # This uses :class:`~optuna.study.Study.best_trial`, which returns the `best_trial` as a
  90 # :class:`~optuna.trial.FrozenTrial`.
  91 # The :class:`~optuna.trial.FrozenTrial` is different from an active trial
  92 # and behaves differently from :class:`~optuna.trial.Trial` in some situations.
  93 # For example, pruning does not work because :class:`~optuna.trial.FrozenTrial.should_prune`
  94 # always returns ``False``.
  95
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/tutorial/20_recipes/010_reuse_best_trial.py b/tutorial/20_recipes/010_reuse_best_trial.py
  --- a/tutorial/20_recipes/010_reuse_best_trial.py
  +++ b/tutorial/20_recipes/010_reuse_best_trial.py
  @@ -1,8 +1,8 @@
   """
   .. _reuse_best_trial:
   
  -Re-use the best values
  -==========================
  +Re-use the best trial
  +======================
   
   In some cases, you may want to re-evaluate the objective function with the best
   hyperparameters again after the hyperparameter optimization.
  @@ -92,3 +92,8 @@
   # and behaves differently from :class:`~optuna.trial.Trial` in some situations.
   # For example, pruning does not work because :class:`~optuna.trial.FrozenTrial.should_prune`
   # always returns ``False``.
  +#
  +# .. note::
  +#     For multi-objective optimization as demonstrated by :ref:`multi_objective`,
  +#     :attr:`~optuna.study.Study.best_trials` returns a list of :class:`~optuna.trial.FrozenTrial`
  +#     on Pareto front. So we can re-use each trial in the list by the similar way above.
  
verification_info: {"golden_diff": "diff --git a/tutorial/20_recipes/010_reuse_best_trial.py b/tutorial/20_recipes/010_reuse_best_trial.py\n--- a/tutorial/20_recipes/010_reuse_best_trial.py\n+++ b/tutorial/20_recipes/010_reuse_best_trial.py\n@@ -1,8 +1,8 @@\n \"\"\"\n .. _reuse_best_trial:\n \n-Re-use the best values\n-==========================\n+Re-use the best trial\n+======================\n \n In some cases, you may want to re-evaluate the objective function with the best\n hyperparameters again after the hyperparameter optimization.\n@@ -92,3 +92,8 @@\n # and behaves differently from :class:`~optuna.trial.Trial` in some situations.\n # For example, pruning does not work because :class:`~optuna.trial.FrozenTrial.should_prune`\n # always returns ``False``.\n+#\n+# .. note::\n+#     For multi-objective optimization as demonstrated by :ref:`multi_objective`,\n+#     :attr:`~optuna.study.Study.best_trials` returns a list of :class:`~optuna.trial.FrozenTrial`\n+#     on Pareto front. So we can re-use each trial in the list by the similar way above.\n", "issue": "Improve the `Re-use the best values` tutorial page\n### What is an issue?\r\n\r\nThere are some minor issues on the Re-use the best values tutorial page.\r\n\r\n## Description\r\n- [x] On the [Re-use the best values](https://optuna.readthedocs.io/en/stable/tutorial/20_recipes/010_reuse_best_trial.html) page, We should update the title as `re-use the best parameters` instead of `re-use the best values`. As the best value means the best objective value in most of the Optuna references.\r\n- [ ] Mention `Study.best_trials` on the page for multi-objective optimization.\r\n\r\n\r\nContext: https://github.com/optuna/optuna/pull/3396#pullrequestreview-917028874\n", "before_files": [{"content": "\"\"\"\n.. _reuse_best_trial:\n\nRe-use the best values\n==========================\n\nIn some cases, you may want to re-evaluate the objective function with the best\nhyperparameters again after the hyperparameter optimization.\n\nFor example,\n\n- You have found good hyperparameters with Optuna and want to run a similar `objective` function using the best hyperparameters found so far to further analyze the results, or\n- You have optimized with Optuna using a partial dataset to reduce training time. After the hyperparameter tuning, you want to train the model using the whole dataset with the best hyperparameter values found.\n\n:class:`~optuna.study.Study.best_trial` provides an interface to re-evaluate the objective function with the current best hyperparameter values.\n\nThis tutorial shows an example of how to re-run a different `objective` function with the current best values, like the first example above.\n\n\nInvestigating the best model further\n-------------------------------------\n\nLet's consider a classical supervised classification problem with Optuna as follows:\n\"\"\"\n\nfrom sklearn import metrics\nfrom sklearn.datasets import make_classification\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\n\n\nimport optuna\n\n\ndef objective(trial):\n    X, y = make_classification(n_features=10, random_state=1)\n    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n\n    C = trial.suggest_float(\"C\", 1e-7, 10.0, log=True)\n\n    clf = LogisticRegression(C=C)\n    clf.fit(X_train, y_train)\n\n    return clf.score(X_test, y_test)\n\n\nstudy = optuna.create_study(direction=\"maximize\")\nstudy.optimize(objective, n_trials=10)\n\nprint(study.best_trial.value)  # Show the best value.\n\n###################################################################################################\n# Suppose after the hyperparameter optimization, you want to calculate other evaluation metrics\n# such as recall, precision, and f1-score on the same dataset.\n# You can define another objective function that shares most of the ``objective``\n# function to reproduce the model with the best hyperparameters.\n\n\ndef detailed_objective(trial):\n    # Use same code objective to reproduce the best model\n    X, y = make_classification(n_features=10, random_state=1)\n    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n\n    C = trial.suggest_float(\"C\", 1e-7, 10.0, log=True)\n\n    clf = LogisticRegression(C=C)\n    clf.fit(X_train, y_train)\n\n    # calculate more evaluation metrics\n    pred = clf.predict(X_test)\n\n    acc = metrics.accuracy_score(pred, y_test)\n    recall = metrics.recall_score(pred, y_test)\n    precision = metrics.precision_score(pred, y_test)\n    f1 = metrics.f1_score(pred, y_test)\n\n    return acc, f1, recall, precision\n\n\n###################################################################################################\n# Pass ``study.best_trial`` as the argument of ``detailed_objective``.\n\ndetailed_objective(study.best_trial)  # calculate acc, f1, recall, and precision\n\n###################################################################################################\n# The difference between :class:`~optuna.study.Study.best_trial` and ordinal trials\n# ----------------------------------------------------------------------------------\n#\n# This uses :class:`~optuna.study.Study.best_trial`, which returns the `best_trial` as a\n# :class:`~optuna.trial.FrozenTrial`.\n# The :class:`~optuna.trial.FrozenTrial` is different from an active trial\n# and behaves differently from :class:`~optuna.trial.Trial` in some situations.\n# For example, pruning does not work because :class:`~optuna.trial.FrozenTrial.should_prune`\n# always returns ``False``.\n", "path": "tutorial/20_recipes/010_reuse_best_trial.py"}], "after_files": [{"content": "\"\"\"\n.. _reuse_best_trial:\n\nRe-use the best trial\n======================\n\nIn some cases, you may want to re-evaluate the objective function with the best\nhyperparameters again after the hyperparameter optimization.\n\nFor example,\n\n- You have found good hyperparameters with Optuna and want to run a similar `objective` function using the best hyperparameters found so far to further analyze the results, or\n- You have optimized with Optuna using a partial dataset to reduce training time. After the hyperparameter tuning, you want to train the model using the whole dataset with the best hyperparameter values found.\n\n:class:`~optuna.study.Study.best_trial` provides an interface to re-evaluate the objective function with the current best hyperparameter values.\n\nThis tutorial shows an example of how to re-run a different `objective` function with the current best values, like the first example above.\n\n\nInvestigating the best model further\n-------------------------------------\n\nLet's consider a classical supervised classification problem with Optuna as follows:\n\"\"\"\n\nfrom sklearn import metrics\nfrom sklearn.datasets import make_classification\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\n\n\nimport optuna\n\n\ndef objective(trial):\n    X, y = make_classification(n_features=10, random_state=1)\n    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n\n    C = trial.suggest_float(\"C\", 1e-7, 10.0, log=True)\n\n    clf = LogisticRegression(C=C)\n    clf.fit(X_train, y_train)\n\n    return clf.score(X_test, y_test)\n\n\nstudy = optuna.create_study(direction=\"maximize\")\nstudy.optimize(objective, n_trials=10)\n\nprint(study.best_trial.value)  # Show the best value.\n\n###################################################################################################\n# Suppose after the hyperparameter optimization, you want to calculate other evaluation metrics\n# such as recall, precision, and f1-score on the same dataset.\n# You can define another objective function that shares most of the ``objective``\n# function to reproduce the model with the best hyperparameters.\n\n\ndef detailed_objective(trial):\n    # Use same code objective to reproduce the best model\n    X, y = make_classification(n_features=10, random_state=1)\n    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n\n    C = trial.suggest_float(\"C\", 1e-7, 10.0, log=True)\n\n    clf = LogisticRegression(C=C)\n    clf.fit(X_train, y_train)\n\n    # calculate more evaluation metrics\n    pred = clf.predict(X_test)\n\n    acc = metrics.accuracy_score(pred, y_test)\n    recall = metrics.recall_score(pred, y_test)\n    precision = metrics.precision_score(pred, y_test)\n    f1 = metrics.f1_score(pred, y_test)\n\n    return acc, f1, recall, precision\n\n\n###################################################################################################\n# Pass ``study.best_trial`` as the argument of ``detailed_objective``.\n\ndetailed_objective(study.best_trial)  # calculate acc, f1, recall, and precision\n\n###################################################################################################\n# The difference between :class:`~optuna.study.Study.best_trial` and ordinal trials\n# ----------------------------------------------------------------------------------\n#\n# This uses :class:`~optuna.study.Study.best_trial`, which returns the `best_trial` as a\n# :class:`~optuna.trial.FrozenTrial`.\n# The :class:`~optuna.trial.FrozenTrial` is different from an active trial\n# and behaves differently from :class:`~optuna.trial.Trial` in some situations.\n# For example, pruning does not work because :class:`~optuna.trial.FrozenTrial.should_prune`\n# always returns ``False``.\n#\n# .. note::\n#     For multi-objective optimization as demonstrated by :ref:`multi_objective`,\n#     :attr:`~optuna.study.Study.best_trials` returns a list of :class:`~optuna.trial.FrozenTrial`\n#     on Pareto front. So we can re-use each trial in the list by the similar way above.\n", "path": "tutorial/20_recipes/010_reuse_best_trial.py"}]}