{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n\n# Lightweight, versatile, and platform agnostic architecture\n\nOptuna is entirely written in Python and has few dependencies.\nThis means that we can quickly move to the real example once you get interested in Optuna.\n\n\n## Quadratic Function Example\n\nUsually, Optuna is used to optimize hyperparameters, but as an example,\nlet's optimize a simple quadratic function: $(x - 2)^2$.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "First of all, import :mod:`optuna`.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import optuna"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "In optuna, conventionally functions to be optimized are named `objective`.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "def objective(trial):\n    x = trial.suggest_float(\"x\", -10, 10)\n    return (x - 2) ** 2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "This function returns the value of $(x - 2)^2$. Our goal is to find the value of ``x``\nthat minimizes the output of the ``objective`` function. This is the \"optimization.\"\nDuring the optimization, Optuna repeatedly calls and evaluates the objective function with\ndifferent values of ``x``.\n\nA :class:`~optuna.trial.Trial` object corresponds to a single execution of the objective\nfunction and is internally instantiated upon each invocation of the function.\n\nThe `suggest` APIs (for example, :func:`~optuna.trial.Trial.suggest_float`) are called\ninside the objective function to obtain parameters for a trial.\n:func:`~optuna.trial.Trial.suggest_float` selects parameters uniformly within the range\nprovided. In our example, from $-10$ to $10$.\n\nTo start the optimization, we create a study object and pass the objective function to method\n:func:`~optuna.study.Study.optimize` as follows.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study = optuna.create_study()\nstudy.optimize(objective, n_trials=100)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You can get the best parameter as follows.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "best_params = study.best_params\nfound_x = best_params[\"x\"]\nprint(\"Found x: {}, (x - 2)^2: {}\".format(found_x, (found_x - 2) ** 2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We can see that the ``x`` value found by Optuna is close to the optimal value of ``2``.\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<div class=\"alert alert-info\"><h4>Note</h4><p>When used to search for hyperparameters in machine learning,\n    usually the objective function would return the loss or accuracy\n    of the model.</p></div>\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Study Object\n\nLet us clarify the terminology in Optuna as follows:\n\n* **Trial**: A single call of the objective function\n* **Study**: An optimization session, which is a set of trials\n* **Parameter**: A variable whose value is to be optimized, such as ``x`` in the above example\n\nIn Optuna, we use the study object to manage optimization.\nMethod :func:`~optuna.study.create_study` returns a study object.\nA study object has useful properties for analyzing the optimization outcome.\n\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the dictionary of parameter name and parameter values:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.best_params"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the best observed value of the objective function:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.best_value"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the best trial:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.best_trial"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get all trials:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.trials"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the number of trials:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "len(study.trials)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "By executing :func:`~optuna.study.Study.optimize` again, we can continue the optimization.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "study.optimize(objective, n_trials=100)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To get the updated number of trials:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "len(study.trials)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "As the objective function is so easy that the last 100 trials don't improve the result.\nHowever, we can check the result again:\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "best_params = study.best_params\nfound_x = best_params[\"x\"]\nprint(\"Found x: {}, (x - 2)^2: {}\".format(found_x, (found_x - 2) ** 2))"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}