Docs updates for kernel and acq func

Author: madeline-scyphers

docs/user_guide/customizing_gp_acq.md

@@ -3,7 +3,7 @@
 BOA is designed with flexibility of options for selecting the acquisition function and
 kernel. BOA could be used by advanced BO users that want to control detailed aspects of the
 optimization. However, for non-domain experts of BO, BOA defaults to common sensible
-choices. BOA defers to BoTorch's defaults of a Matern 5/2 kernel, one of the most widely
+choices. BOA defers to BoTorch's defaults of a Matern 5/2 kernel, one of the most widely
 used and flexible choices for BO and GPs (Frazier, 2018; Riche and Picheny, 2021; Jakeman,
 2023). This is considered to be a flexible and broadly applicable kernel (Riche and Picheny,2021) and it is used as the default by many other BO and GP toolkits (Akiba et al., 2019;
 Balandat et al., 2020; Brea, 2023; Nogueira, 2014; Jakeman, 2023). Similarly, BOA defaults
@@ -38,6 +38,61 @@ generation_strategy:
 
 ## Utilizing Ax's Predefined Kernels and Acquisition Functions
 
+Ax has a number of predefined kernels and acquisition function combos that can be used in the optimization process. Each of these sit inside a "step" inside the generation strategy, where your optimization is broken into a number of "steps" and each step can have its own kernel and acquisition function. For example, the first step is usually a Sobol step that does a quasi-random initialization of the optimization process. The second step could be a "GPEI" step (GPEI is the Ax model class name, and is the default used for single objective optimization) that uses the Matern 5/2 kernel and the batched noisy Expected Improvement acquisition function.
+
+```yaml
+
+generation_strategy:
+    steps:
+        - model: Sobol
+          num_trials: 50
+          # specify the maximum number of trials to run in parallel
+          # 
+          max_parallelism: 10
+        - model: GPEI
+          num_trials: -1  # -1 means the rest of the trials
+          max_parallelism: 10
+``` 
+
+Ax does not have a good spot in their docs currently that lists all the available kernels and acquisition functions combo models, but you can find them listed on their [api docs here](https://ax.dev/api/modelbridge.html#ax.modelbridge.registry.Models) and you can see the source code for the models by clicking the source link on the api docs page. Some of the available models are:
+
+- `GPEI`: Gaussian Process Expected Improvement, the default for single objective optimization, uses the Matern 5/2 kernel
+- `GPKG`: Gaussian Process Knowledge Gradient, uses the Matern 5/2 kernel
+- `SAASBO`: Sparse Axis-Aligned Subspace Bayesian Optimization, see [BO Overview High Dimensionality](bo_overview.md#high-dimensionality) for more details, uses the Matern 5/2 kernel and the batched noisy Expected Improvement acquisition function
+- `Sobol`: Sobol initialization
+- `MOO`: Gaussian Process Expected Hypervolume Improvement, uses the Matern 5/2 kernel
+
+If you want to specify your kernel and acquisition function, you can do so by creating a custom model. The way to do that is with the `BOTORCH_MODULAR` model. This model allows you to specify the kernel and acquisition function you want to use. Here is an example of how to use the `BOTORCH_MODULAR` model:
+
+```yaml
+generation_strategy:
+    steps:
+        - model: SOBOL
+          num_trials: 5
+        - model: BOTORCH_MODULAR
+          num_trials: -1  # No limitation on how many trials should be produced from this step
+          model_kwargs:
+              surrogate:
+                  botorch_model_class: SingleTaskGP  # BoTorch model class name
+                  covar_module_class: RBFKernel  # GPyTorch kernel class name
+                  mll_class: LeaveOneOutPseudoLikelihood  # GPyTorch MarginalLogLikelihood class name
+              botorch_acqf_class: qUpperConfidenceBound  # BoTorch acquisition function class name
+              acquisition_options:
+                  beta: 0.5
+``` 
+
+In the above example, the `BOTORCH_MODULAR` model is used to specify the `SingleTaskGP` model class, the `RBFKernel` kernel class, and the `qUpperConfidenceBound` acquisition function class. The `qUpperConfidenceBound` acquisition function is a batched version of UpperConfidenceBound. The `beta` parameter is a hyperparameter of the acquisition function that controls the trade-off between exploration and exploitation.
+
+BoTorch model classes can be found in the [BoTorch model api documentation](https://botorch.org/docs/models) and the BoTorch acquisition functions can be found in the [BoTorch acquisition api documentation](https://botorch.org/api/acquisition.html).
+
+GPyTorch kernel classes can be found in the [GPyTorch kernel api documentation](https://gpytorch.readthedocs.io/en/latest/kernels.html).
+
+The GPyTorch MarginalLogLikelihood classes can be found in the [GPyTorch MarginalLogLikelihood api documentation](https://gpytorch.readthedocs.io/en/latest/marginal_log_likelihoods.html). But the only MLL class that for sure work currently are `ExactMarginalLogLikelihood` and `LeaveOneOutPseudoLikelihood`. Other MLL classes may work, but they have not been tested and are depended on some other implementation details in Ax.
+
+
+```{caution}
+The `BOTORCH_MODULAR` class is an area of Ax's code that is still under active development and a lot of components of it are very dependent on the current implementation of Ax, BoTorch, and GPyTorch, and therefore it is impossible to test every possible combination of kernel and acquisition function. Therefore, it is recommended to use when possible the predefined models that Ax provides.
+```
 
 
 

docs/user_guide/package_overview.rst

@@ -26,8 +26,6 @@ Objective functions
 
 When specifying your objective function to minimize or maximize, :doc:`BOA </index>` comes with a number of metrics you can use with your model output, such as MSE, :math:`R^2`, and others. For a list of current list of premade available of metrics, see See :mod:`.metrics.metrics`
 
-
-
 ************************************************************************
 Creating a model wrapper (Language Agnostic or Python API)
 ************************************************************************
@@ -39,6 +37,17 @@ and there is a standard interface to follow.
 
 
 See the :mod:`instructions for creating a model wrapper <.boa.wrappers>` for details.
+See the :doc:`examples of model wrappers <.boa.wrappers>` for examples.
+See :doc:`tutorials </examples/index>`  for a number of examples of model wrappers in both Python and R.
+
+
+************************************************************************
+Choosing a Custom Kernel and Acquisition Function
+************************************************************************
+
+BOA tries to make it easy to use the default kernel and acquisition function, but if you need to specify a different kernel or acquisition function, you can do so in the configuration file.
+
+See :doc:`details on how to specify kernel and acquisition function <customizing_gp_acq.>` for details.
 
 ****************************************************
 Creating a Python launch script (Usually Not Needed)

tests/scripts/other_langs/r_package_streamlined/config_modular_botorch.yaml

@@ -54,7 +54,7 @@ generation_strategy:
                   botorch_model_class: SingleTaskGP  # BoTorch model class name
 
                   covar_module_class: RBFKernel  # GPyTorch kernel class name
-                  mll_class: LeaveOneOutPseudoLikelihood
+                  mll_class: LeaveOneOutPseudoLikelihood  # GPyTorch MarginalLogLikelihood class name
               botorch_acqf_class: qUpperConfidenceBound  # BoTorch acquisition function class name
               acquisition_options:
                   beta: 0.5