Pushing the docs for revision for branch: master, commit 39d7053223b583c97df53ab6881d6f4508922435

Author: skoptci

beta/doctrees/auto_examples/ask-and-tell.doctree

@@ -15,7 +15,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "\n# Parallel optimization\n\n\nIaroslav Shcherbatyi, May 2017.\n\nReviewed by Manoj Kumar and Tim Head.\n\nReformatted by Holger Nahrstaedt 2020\n\n.. currentmodule:: skopt\n\nIntroduction\n============\n\nFor many practical black box optimization problems expensive objective can be\nevaluated in parallel at multiple points. This allows to get more objective\nevaluations per unit of time, which reduces the time necessary to reach good\nobjective values when appropriate optimization algorithms are used, see for\nexample results in [1] and the references therein.\n\n\nOne such example task is a selection of number and activation function of a\nneural network which results in highest accuracy for some machine learning\nproblem. For such task, multiple neural networks with different combinations\nof number of neurons and activation function type can be evaluated at the same\ntime in parallel on different cpu cores / computational nodes.\n\nThe \u201cask and tell\u201d API of scikit-optimize exposes functionality that allows to\nobtain multiple points for evaluation in parallel. Intended usage of this\ninterface is as follows:\n\n1. Initialize instance of the `Optimizer` class from skopt\n2. Obtain n points for evaluation in parallel by calling the `ask` method of\n    an optimizer instance with the `n_points` argument set to n > 0\n3. Evaluate points\n4. Provide points and corresponding objectives using the `tell` method of\n    an optimizer instance\n5. Continue from step 2 until eg maximum number of evaluations reached\n"
+        "\n# Parallel optimization\n\n\nIaroslav Shcherbatyi, May 2017.\nReviewed by Manoj Kumar and Tim Head.\nReformatted by Holger Nahrstaedt 2020\n\n.. currentmodule:: skopt\n\nIntroduction\n============\n\nFor many practical black box optimization problems expensive objective can be\nevaluated in parallel at multiple points. This allows to get more objective\nevaluations per unit of time, which reduces the time necessary to reach good\nobjective values when appropriate optimization algorithms are used, see for\nexample results in [1]_ and the references therein.\n\n\nOne such example task is a selection of number and activation function of a\nneural network which results in highest accuracy for some machine learning\nproblem. For such task, multiple neural networks with different combinations\nof number of neurons and activation function type can be evaluated at the same\ntime in parallel on different cpu cores / computational nodes.\n\nThe \u201cask and tell\u201d API of scikit-optimize exposes functionality that allows to\nobtain multiple points for evaluation in parallel. Intended usage of this\ninterface is as follows:\n\n1. Initialize instance of the `Optimizer` class from skopt\n2. Obtain n points for evaluation in parallel by calling the `ask` method of an optimizer instance with the `n_points` argument set to n > 0\n3. Evaluate points\n4. Provide points and corresponding objectives using the `tell` method of an optimizer instance\n5. Continue from step 2 until eg maximum number of evaluations reached\n"
       ]
     },
     {
@@ -51,7 +51,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "Note that if `n_points` is set to some integer > 0 for the `ask` method, the\nresult will be a list of points, even for `n_points`=1. If the argument is\nset to `None` (default value) then a single point (but not a list of points)\nwill be returned.\n\nThe default \"minimum constant liar\" [1] parallelization strategy is used in\nthe example, which allows to obtain multiple points for evaluation with a\nsingle call to the `ask` method with any surrogate or acquisition function.\nParalellization strategy can be set using the \"strategy\" argument of `ask`.\nFor supported parallelization strategies see the documentation of\nscikit-optimize.\n\n[1] [https://hal.archives-ouvertes.fr/hal-00732512/document](https://hal.archives-ouvertes.fr/hal-00732512/document) .\n"
+        "Note that if `n_points` is set to some integer > 0 for the `ask` method, the\nresult will be a list of points, even for `n_points` = 1. If the argument is\nset to `None` (default value) then a single point (but not a list of points)\nwill be returned.\n\nThe default \"minimum constant liar\" [1]_ parallelization strategy is used in\nthe example, which allows to obtain multiple points for evaluation with a\nsingle call to the `ask` method with any surrogate or acquisition function.\nParallelization strategy can be set using the \"strategy\" argument of `ask`.\nFor supported parallelization strategies see the documentation of\nscikit-optimize.\n\n.. [1] `<https://hal.archives-ouvertes.fr/hal-00732512/document>`_\n"
       ]
     }
   ],

beta/doctrees/auto_examples/bayesian-optimization.doctree

@@ -30,7 +30,7 @@
 # Toy model
 # =========
 #
-# We will use the `branin` function as toy model for the expensive function.
+# We will use the :class:`benchmarks.branin` function as toy model for the expensive function.
 # In a real world application this function would be unknown and expensive
 # to evaluate.
 
@@ -84,7 +84,7 @@ def plot_branin():
 #
 # The objective of this example is to find one of these minima in as
 # few iterations as possible. One iteration is defined as one call
-# to the `branin` function.
+# to the :class:`benchmarks.branin` function.
 #
 # We will evaluate each model several times using a different seed for the
 # random number generator. Then compare the average performance of these
@@ -133,12 +133,12 @@ def run(minimizer, n_iter=5):
 #############################################################################
 # This plot shows the value of the minimum found (y axis) as a function
 # of the number of iterations performed so far (x axis). The dashed red line
-# indicates the true value of the minimum of the branin function.
+# indicates the true value of the minimum of the :class:`benchmarks.branin` function.
 #
 # For the first ten iterations all methods perform equally well as they all
 # start by creating ten random samples before fitting their respective model
 # for the first time. After iteration ten the next point at which
-# to evaluate `branin` is guided by the model, which is where differences
+# to evaluate :class:`benchmarks.branin` is guided by the model, which is where differences
 # start to appear.
 #
 # Each minimizer only has access to noisy observations of the objective

beta/doctrees/auto_examples/hyperparameter-optimization.doctree

@@ -170,7 +170,7 @@
       },
       "outputs": [],
       "source": [
-        "for i in range(50):\n    next_x = opt.ask()\n    f_val = objective(next_x)\n    opt.tell(next_x, f_val)\n\nplot_optimizer(opt, x, fx)"
+        "for i in range(10):\n    next_x = opt.ask()\n    f_val = objective(next_x)\n    opt.tell(next_x, f_val)\n\nplot_optimizer(opt, x, fx)"
       ]
     },
     {

beta/doctrees/auto_examples/index.doctree

@@ -141,7 +141,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "Let us now visually examine\n\n1. The approximation of the fit gp model to the original function.\n2. The acquistion values that determine the next point to be queried.\n\n"
+        "Let us now visually examine\n\n1. The approximation of the fit gp model to the original function.\n2. The acquisition values that determine the next point to be queried.\n\n"
       ]
     },
     {

beta/doctrees/auto_examples/interruptible-optimization.doctree

@@ -19,16 +19,16 @@
  will be spread over the whole parameter space and later samples should
  cluster around the minimum.
 
-The `plot_evaluations()` function helps with visualizing the location and
+The :class:`plots.plot_evaluations` function helps with visualizing the location and
 order in which samples are evaluated for objectives with an arbitrary
 number of dimensions.
 
-The `plot_objective()` function plots the partial dependence of the objective,
+The :class:`plots.plot_objective` function plots the partial dependence of the objective,
 as represented by the surrogate model, for each dimension and as pairs of the
 input dimensions.
 
 All of the minimizers implemented in `skopt` return an [`OptimizeResult`]()
-instance that can be inspected. Both `plot_evaluations` and `plot_objective`
+instance that can be inspected. Both :class:`plots.plot_evaluations` and :class:`plots.plot_objective`
 are helpers that do just that
 """
 
@@ -42,15 +42,15 @@
 # Toy models
 # ==========
 #
-# We will use two different toy models to demonstrate how `plot_evaluations()`
+# We will use two different toy models to demonstrate how :class:`plots.plot_evaluations`
 # works.
 #
-# The first model is the `branin` function which has two dimensions and three
+# The first model is the :class:`benchmarks.branin` function which has two dimensions and three
 # minima.
 #
 # The second model is the `hart6` function which has six dimension which makes
 # it hard to visualize. This will show off the utility of
-# `plot_evaluations()`.
+# :class:`plots.plot_evaluations`.
 
 from skopt.benchmarks import branin as branin
 from skopt.benchmarks import hart6 as hart6_
@@ -64,7 +64,7 @@ def hart6(x):
 # Starting with `branin`
 # ======================
 #
-# To start let's take advantage of the fact that `branin` is a simple
+# To start let's take advantage of the fact that :class:`benchmarks.branin` is a simple
 # function which can be visualised in two dimensions.
 
 from matplotlib.colors import LogNorm
@@ -105,8 +105,8 @@ def plot_branin():
 # =================================
 #
 # Next we use an extra trees based minimizer to find one of the minima of the
-# `branin` function. Then we visualize at which points the objective is being
-# evaluated using `plot_evaluations()`.
+# :class:`benchmarks.branin` function. Then we visualize at which points the objective is being
+# evaluated using :class:`plots.plot_evaluations`.
 
 from functools import partial
 from skopt.plots import plot_evaluations
@@ -122,7 +122,7 @@ def plot_branin():
 _ = plot_evaluations(forest_res, bins=10)
 
 #############################################################################
-# `plot_evaluations()` creates a grid of size `n_dims` by `n_dims`.
+# :class:`plots.plot_evaluations` creates a grid of size `n_dims` by `n_dims`.
 # The diagonal shows histograms for each of the dimensions. In the lower
 # triangle (just one plot in this case) a two dimensional scatter plot of all
 # points is shown. The order in which points were evaluated is encoded in the
@@ -134,7 +134,7 @@ def plot_branin():
 # of the true miminum. The histograms show that the objective is evaluated
 # more often at locations near to one of the three minima.
 #
-# Using `plot_objective()` we can visualise the one dimensional partial
+# Using :class:`plots.plot_objective` we can visualise the one dimensional partial
 # dependence of the surrogate model for each dimension. The contour plot in
 # the bottom left corner shows the two dimensional partial dependence. In this
 # case this is the same as simply plotting the objective as it only has two
@@ -144,8 +144,7 @@ def plot_branin():
 # ------------------------
 #
 # Partial dependence plots were proposed by
-# [Friedman (2001)](http://projecteuclid.org/euclid.aos/1013203451)
-# (doi:10.1214/aos/1013203451 section 8.2)
+# [Friedman (2001)]_
 # as a method for interpreting the importance of input features used in
 # gradient boosting machines. Given a function of :math:`k`: variables
 # :math:`y=f\left(x_1, x_2, ..., x_k\right)`: the
@@ -154,7 +153,7 @@ def plot_branin():
 # with the sum running over a set of $N$ points drawn at random from the
 # search space.
 #
-# The idea is to visulize how the value of :math:`x_j`: influences the function
+# The idea is to visualize how the value of :math:`x_j`: influences the function
 # :math:`f`: after averaging out the influence of all other variables.
 
 from skopt.plots import plot_objective
@@ -171,7 +170,7 @@ def plot_branin():
 # Random sampling
 # ===============
 #
-# Compare this to a mimizer which picks points at random. There is no
+# Compare this to a minimizer which picks points at random. There is no
 # structure visible in the order in which it evaluates the objective. Because
 # there is no model involved in the process of picking sample points at
 # random, we can not plot the partial dependence of the model.
@@ -185,12 +184,12 @@ def plot_branin():
 # =========================
 #
 # Visualising what happens in two dimensions is easy, where
-# `plot_evaluations()` and `plot_objective()` start to be useful is when the
+# :class:`plots.plot_evaluations` and :class:`plots.plot_objective` start to be useful is when the
 # number of dimensions grows. They take care of many of the more mundane
 # things needed to make good plots of all combinations of the dimensions.
 #
-# The next example uses `hart6` which has six dimensions and shows both
-# `plot_evaluations()` and `plot_objective()`.
+# The next example uses class:`benchmarks.hart6` which has six dimensions and shows both
+# :class:`plots.plot_evaluations` and :class:`plots.plot_objective`.
 
 bounds = [(0., 1.),] * 6
 
@@ -207,7 +206,7 @@ def plot_branin():
 # ==============================
 #
 # To make things more interesting let's add two dimension to the problem.
-# As `hart6` only depends on six dimensions we know that for this problem
+# As :class:`benchmarks.hart6` only depends on six dimensions we know that for this problem
 # the new dimensions will be "flat" or uninformative. This is clearly visible
 # in both the placement of samples and the partial dependence plots.
 
@@ -221,5 +220,4 @@ def plot_branin():
 _ = plot_evaluations(forest_res)
 _ = plot_objective(forest_res)
 
-
-
+# .. [Friedman (2001)] `doi:10.1214/aos/1013203451 section 8.2 <http://projecteuclid.org/euclid.aos/1013203451>`

beta/doctrees/auto_examples/parallel-optimization.doctree

@@ -14,18 +14,15 @@
 If for some reason the process has to be interrupted results are irreversibly
 lost, and the routine has to start over from the beginning.
 
-With the help of the `CheckpointSaver` callback the optimizer's current state
+With the help of the :class:`callbacks.CheckpointSaver` callback the optimizer's current state
 can be saved after each iteration, allowing to restart from that point at any
 time.
 
 This is useful, for example,
 
-* if you don't know how long the process will take and cannot hog
-    computational resources forever
-* if there might be system failures due to shaky infrastructure
-    (or colleagues...)
-* if you want to adjust some parameters and continue with the already obtained
-    results
+* if you don't know how long the process will take and cannot hog computational resources forever
+* if there might be system failures due to shaky infrastructure (or colleagues...)
+* if you want to adjust some parameters and continue with the already obtained results
 
 """
 print(__doc__)
@@ -45,7 +42,7 @@
 #
 # We will use pretty much the same optimization problem as in the
 # :ref:`sphx_glr_auto_examples_bayesian-optimization.py`
-# notebook. Additionaly we will instantiate the `CheckpointSaver`
+# notebook. Additionally we will instantiate the :class:`callbacks.CheckpointSaver`
 # and pass it to the minimizer:
 
 from skopt import gp_minimize
@@ -82,14 +79,14 @@ def obj_fun(x, noise_level=noise_level):
 # tells you that he had had an update scheduled for Sunday noon – who
 # doesn't like updates?
 #
-# `gp_minimize` did not finish, and there is no `res` variable with the
+# :class:`gp_minimize` did not finish, and there is no `res` variable with the
 # actual results!
 #
 # Restoring the last checkpoint
 # =============================
 #
-# Luckily we employed the `CheckpointSaver` and can now restore the latest
-# result with `skopt.load`
+# Luckily we employed the :class:`callbacks.CheckpointSaver` and can now restore the latest
+# result with :class:`skopt.load`
 # (see :ref:`sphx_glr_auto_examples_store-and-load-results.py` for more
 # information on that)
 
@@ -123,10 +120,11 @@ def obj_fun(x, noise_level=noise_level):
 # =================
 #
 # * **changes in search space:** You can use this technique to interrupt
-# the search, tune the search space and continue the optimization. Note
-# that the optimizers will complain if `x0` contains parameter values not
-# covered by the dimension definitions, so in many cases shrinking the
-# search space will not work without deleting the offending runs from
-# `x0` and `y0`.
+#   the search, tune the search space and continue the optimization. Note
+#   that the optimizers will complain if `x0` contains parameter values not
+#   covered by the dimension definitions, so in many cases shrinking the
+#   search space will not work without deleting the offending runs from
+#   `x0` and `y0`.
 # * see :ref:`sphx_glr_auto_examples_store-and-load-results.py`
+#
 # for more information on how the results get saved and possible caveats

beta/doctrees/auto_examples/sg_execution_times.doctree

@@ -11,19 +11,19 @@
 ============
 
 This example assumes basic familiarity with
-`scikit-learn <http://scikit-learn.org/stable/index.html>`.
+`scikit-learn <http://scikit-learn.org/stable/index.html>`_.
 
 Search for parameters of machine learning models that result in best
 cross-validation performance is necessary in almost all practical
 cases to get a model with best generalization estimate. A standard
-approach in scikit-learn is using `GridSearchCV` class, which takes
+approach in scikit-learn is using :obj:`sklearn.model_selection.GridSearchCV` class, which takes
 a set of values for every parameter to try, and simply enumerates all
 combinations of parameter values. The complexity of such search grows
 exponentially with the addition of new parameters. A more scalable
-approach is using `RandomizedSearchCV`, which however does not take
+approach is using :obj:`sklearn.model_selection.RandomizedSearchCV`, which however does not take
 advantage of the structure of a search space.
 
-Scikit-optimize provides a drop-in replacement for `GridSearchCV`,
+Scikit-optimize provides a drop-in replacement for :obj:`sklearn.model_selection.GridSearchCV`,
 which utilizes Bayesian Optimization where a predictive model referred
 to as "surrogate" is used to model the search space and utilized to
 arrive at good parameter values combination as soon as possible.
@@ -126,7 +126,7 @@
 # Progress monitoring and control using `callback` argument of `fit` method
 # =========================================================================
 #
-# It is possible to monitor the progress of BayesSearchCV with an event
+# It is possible to monitor the progress of :class:`BayesSearchCV` with an event
 # handler that is called on every step of subspace exploration. For single job
 # mode, this is called on every evaluation of model configuration, and for
 # parallel mode, this is called when n_jobs model configurations are evaluated

beta/doctrees/auto_examples/sklearn-gridsearchcv-replacement.doctree

@@ -146,7 +146,7 @@ def plot_optimizer(opt, x, fx):
 # Let us sample a few more points and plot the optimizer again:
 
 
-for i in range(50):
+for i in range(10):
     next_x = opt.ask()
     f_val = objective(next_x)
     opt.tell(next_x, f_val)