{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n\n# Re-use the best values\n\nIn some cases, you may want to re-evaluate the objective function with the best\nhyperparameters again after the hyperparameter optimization.\n\nFor example,\n\n- You have found good hyperparameters with Optuna and want to run a similar `objective` function using the best hyperparameters found so far to further analyze the results, or\n- You have optimized with Optuna using a partial dataset to reduce training time. After the hyperparameter tuning, you want to train the model using the whole dataset with the best hyperparameter values found.\n\n:class:`~optuna.study.Study.best_trial` provides an interface to re-evaluate the objective function with the current best hyperparameter values.\n\nThis tutorial shows an example of how to re-run a different `objective` function with the current best values, like the first example above.\n\n\n## Investigating the best model further\n\nLet's consider a classical supervised classification problem with Optuna as follows:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from sklearn import metrics\nfrom sklearn.datasets import make_classification\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.model_selection import train_test_split\n\n\nimport optuna\n\n\ndef objective(trial):\n    X, y = make_classification(n_features=10, random_state=1)\n    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n\n    C = trial.suggest_loguniform(\"C\", 1e-7, 10.0)\n\n    clf = LogisticRegression(C=C)\n    clf.fit(X_train, y_train)\n\n    return clf.score(X_test, y_test)\n\n\nstudy = optuna.create_study(direction=\"maximize\")\nstudy.optimize(objective, n_trials=10)\n\nprint(study.best_trial.value)  # Show the best value."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Suppose after the hyperparameter optimization, you want to calculate other evaluation metrics\nsuch as recall, precision, and f1-score on the same dataset.\nYou can define another objective function that shares most of the ``objective``\nfunction to reproduce the model with the best hyperparameters.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "def detailed_objective(trial):\n    # Use same code objective to reproduce the best model\n    X, y = make_classification(n_features=10, random_state=1)\n    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n\n    C = trial.suggest_loguniform(\"C\", 1e-7, 10.0)\n\n    clf = LogisticRegression(C=C)\n    clf.fit(X_train, y_train)\n\n    # calculate more evaluation metrics\n    pred = clf.predict(X_test)\n\n    acc = metrics.accuracy_score(pred, y_test)\n    recall = metrics.recall_score(pred, y_test)\n    precision = metrics.precision_score(pred, y_test)\n    f1 = metrics.f1_score(pred, y_test)\n\n    return acc, f1, recall, precision"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Pass ``study.best_trial`` as the argument of ``detailed_objective``.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "detailed_objective(study.best_trial)  # calculate acc, f1, recall, and precision"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## The difference between :class:`~optuna.study.Study.best_trial` and ordinal trials\n\nThis uses :class:`~optuna.study.Study.best_trial`, which returns the `best_trial` as a\n:class:`~optuna.trial.FrozenTrial`.\nThe :class:`~optuna.trial.FrozenTrial` is different from an active trial\nand behaves differently from :class:`~optuna.trial.Trial` in some situations.\nFor example, pruning does not work because :class:`~optuna.trial.FrozenTrial.should_prune`\nalways returns ``False``.\n\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}