{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n\n# Easy Parallelization\n\nIt's straightforward to parallelize :func:`optuna.study.Study.optimize`.\n\nIf you want to manually execute Optuna optimization:\n\n    1. start an RDB server (this example uses MySQL)\n    2. create a study with `--storage` argument\n    3. share the study among multiple nodes and processes\n\nOf course, you can use Kubernetes as in [the kubernetes examples](https://github.com/optuna/optuna/tree/master/examples/kubernetes).\n\nTo just see how parallel optimization works in Optuna, check the below video.\n\n.. raw:: html\n\n    <iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/J_aymk4YXhg?start=427\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>\n\n\n## Create a Study\n\nYou can create a study using ``optuna create-study`` command.\nAlternatively, in Python script you can use :func:`optuna.create_study`.\n\n\n```bash\n$ mysql -u root -e \"CREATE DATABASE IF NOT EXISTS example\"\n$ optuna create-study --study-name \"distributed-example\" --storage \"mysql://root@localhost/example\"\n[I 2020-07-21 13:43:39,642] A new study created with name: distributed-example\n```\nThen, write an optimization script. Let's assume that ``foo.py`` contains the following code.\n\n```python\nimport optuna\n\n\ndef objective(trial):\n    x = trial.suggest_float(\"x\", -10, 10)\n    return (x - 2) ** 2\n\n\nif __name__ == \"__main__\":\n    study = optuna.load_study(\n        study_name=\"distributed-example\", storage=\"mysql://root@localhost/example\"\n    )\n    study.optimize(objective, n_trials=100)\n```\n## Share the Study among Multiple Nodes and Processes\n\nFinally, run the shared study from multiple processes.\nFor example, run ``Process 1`` in a terminal, and do ``Process 2`` in another one.\nThey get parameter suggestions based on shared trials' history.\n\nProcess 1:\n\n```bash\n$ python foo.py\n[I 2020-07-21 13:45:02,973] Trial 0 finished with value: 45.35553104173011 and parameters: {'x': 8.73465151598285}. Best is trial 0 with value: 45.35553104173011.\n[I 2020-07-21 13:45:04,013] Trial 2 finished with value: 4.6002397305938905 and parameters: {'x': 4.144816945707463}. Best is trial 1 with value: 0.028194513284051464.\n...\n```\nProcess 2 (the same command as process 1):\n\n```bash\n$ python foo.py\n[I 2020-07-21 13:45:03,748] Trial 1 finished with value: 0.028194513284051464 and parameters: {'x': 1.8320877810162361}. Best is trial 1 with value: 0.028194513284051464.\n[I 2020-07-21 13:45:05,783] Trial 3 finished with value: 24.45966755098074 and parameters: {'x': 6.945671597566982}. Best is trial 1 with value: 0.028194513284051464.\n...\n```\n<div class=\"alert alert-info\"><h4>Note</h4><p>We do not recommend SQLite for distributed optimizations at scale because it may cause deadlocks and serious performance issues. Please consider to use another database engine like PostgreSQL or MySQL.</p></div>\n\n<div class=\"alert alert-info\"><h4>Note</h4><p>Please avoid putting the SQLite database on NFS when running distributed optimizations. See also: https://www.sqlite.org/faq.html#q5</p></div>\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}