|
| 1 | +--- |
| 2 | +title: "Tuning Multiple Similar Parameters: Ensuring `autoplot` Uniqueness" |
| 3 | +output: rmarkdown::html_vignette |
| 4 | +vignette: > |
| 5 | + %\VignetteIndexEntry{Tuning Multiple Similar Parameters: Ensuring `autoplot` Uniqueness} |
| 6 | + %\VignetteEngine{knitr::rmarkdown} |
| 7 | + %\VignetteEncoding{UTF-8} |
| 8 | +--- |
| 9 | + |
| 10 | +```{r setup, include = FALSE} |
| 11 | +knitr::opts_chunk$set( |
| 12 | + collapse = TRUE, |
| 13 | + comment = "#>", |
| 14 | + eval = reticulate::py_module_available("keras") && |
| 15 | + requireNamespace("keras3", quietly = TRUE) && |
| 16 | + requireNamespace("ggplot2", quietly = TRUE) && |
| 17 | + requireNamespace("tune", quietly = TRUE) && |
| 18 | + requireNamespace("dials", quietly = TRUE) && |
| 19 | + requireNamespace("parsnip", quietly = TRUE) && |
| 20 | + requireNamespace("workflows", quietly = TRUE) && |
| 21 | + requireNamespace("recipes", quietly = TRUE) && |
| 22 | + requireNamespace("rsample", quietly = TRUE) |
| 23 | +) |
| 24 | +``` |
| 25 | + |
| 26 | +## Introduction |
| 27 | + |
| 28 | +When using `kerasnip` to define and tune Keras models within the `tidymodels` framework, you might encounter situations where you want to tune multiple parameters that, by default, map to the same underlying `dials` parameter type. A common example is tuning the number of units in multiple `layer_dense` blocks within the same model. |
| 29 | + |
| 30 | +While `kerasnip` intelligently maps these parameters (e.g., `dense1_units` and `dense2_units` both map to `dials::hidden_units()`), this can lead to ambiguity when visualizing tuning results with `ggplot2::autoplot()`. Without a way to distinguish between these otherwise identical parameter types, `autoplot()` may produce errors or misleading plots. |
| 31 | + |
| 32 | +This vignette demonstrates how to explicitly provide unique identifiers to your tuned parameters, ensuring `autoplot()` can correctly visualize the results for each distinct parameter. |
| 33 | + |
| 34 | +## The Problem (Implicit) |
| 35 | + |
| 36 | +Consider a model with two dense layers, each with a `units` parameter. If you were to define them for tuning without unique `id`s, `autoplot()` would encounter an issue because it cannot distinguish between the two parameters. |
| 37 | + |
| 38 | +For example, if you were to run `ggplot2::autoplot(tune_res)` without unique `id`s, you might encounter an error similar to this: |
| 39 | + |
| 40 | +```{r} |
| 41 | +#> Error in `dplyr::rename()`: |
| 42 | +#> ! Names must be unique. |
| 43 | +#> ✖ These names are duplicated: |
| 44 | +#> * "# Hidden Units" at locations 1 and 2. |
| 45 | +``` |
| 46 | + |
| 47 | +This error clearly indicates that `autoplot()` is trying to rename columns for plotting, but it finds duplicate names like "# Hidden Units" because both `dense1_units` and `dense2_units` are generically identified as `hidden_units` by `dials` without further distinction. This makes it impossible for `autoplot()` to differentiate their tuning results. |
| 48 | + |
| 49 | +## The Solution: Using Unique `id`s with `tune()` |
| 50 | + |
| 51 | +The solution is to provide a unique `id` argument to the `tune()` function for each parameter you wish to distinguish. |
| 52 | + |
| 53 | +Let's define a simple sequential Keras model with two dense layers: |
| 54 | + |
| 55 | +```{r model_definition} |
| 56 | +library(kerasnip) |
| 57 | +library(keras3) |
| 58 | +library(parsnip) |
| 59 | +library(dials) |
| 60 | +library(workflows) |
| 61 | +library(recipes) |
| 62 | +library(rsample) |
| 63 | +library(tune) |
| 64 | +library(ggplot2) |
| 65 | +
|
| 66 | +# Define a spec with multiple hidden unit parameters |
| 67 | +model_name <- "autoplot_unique_spec" |
| 68 | +# Clean up the spec if it already exists from a previous run |
| 69 | +if (exists(model_name, mode = "function")) { |
| 70 | + suppressMessages(remove_keras_spec(model_name)) |
| 71 | +} |
| 72 | +
|
| 73 | +input_block <- function(model, input_shape) { |
| 74 | + keras3::keras_model_sequential(input_shape = input_shape) |
| 75 | +} |
| 76 | +
|
| 77 | +dense_block <- function(model, units = 10) { |
| 78 | + model |> keras3::layer_dense(units = units) |
| 79 | +} |
| 80 | +
|
| 81 | +output_block <- function(model, num_classes) { |
| 82 | + model |> |
| 83 | + keras3::layer_dense(units = num_classes, activation = "softmax") |
| 84 | +} |
| 85 | +
|
| 86 | +create_keras_sequential_spec( |
| 87 | + model_name = model_name, |
| 88 | + layer_blocks = list( |
| 89 | + input = input_block, |
| 90 | + dense1 = dense_block, |
| 91 | + dense2 = dense_block, |
| 92 | + output = output_block |
| 93 | + ), |
| 94 | + mode = "classification" |
| 95 | +) |
| 96 | +
|
| 97 | +# Now, create the model specification and assign unique IDs for tuning |
| 98 | +tune_spec <- autoplot_unique_spec( |
| 99 | + dense1_units = tune(id = "dense_layer_one_units"), |
| 100 | + dense2_units = tune(id = "dense_layer_two_units") |
| 101 | +) |> |
| 102 | + set_engine("keras") |
| 103 | +
|
| 104 | +print(tune_spec) |
| 105 | +``` |
| 106 | + |
| 107 | +Notice how `dense1_units` and `dense2_units` are both passed to `tune()`, but each is given a distinct `id`. This `id` acts as a label that `autoplot()` can use to differentiate the parameters. |
| 108 | + |
| 109 | +### Setting up the Tuning Workflow |
| 110 | + |
| 111 | +Next, we'll set up a `tidymodels` workflow, define the parameter ranges, and create a tuning grid. |
| 112 | + |
| 113 | +```{r tuning_setup} |
| 114 | +# Set up workflow and tuning grid |
| 115 | +rec <- recipes::recipe(Species ~ ., data = iris) |
| 116 | +tune_wf <- workflows::workflow(rec, tune_spec) |
| 117 | +
|
| 118 | +params <- tune::extract_parameter_set_dials(tune_wf) |
| 119 | +
|
| 120 | +# Update the parameter ranges using kerasnip::hidden_units |
| 121 | +# The `id`s provided in tune() are automatically detected and used here. |
| 122 | +params <- params |> |
| 123 | + update( |
| 124 | + dense_layer_one_units = hidden_units(range = c(4L, 8L)), |
| 125 | + dense_layer_two_units = hidden_units(range = c(4L, 8L)) |
| 126 | + ) |
| 127 | +
|
| 128 | +grid <- dials::grid_regular(params, levels = 2) |
| 129 | +control <- tune::control_grid(save_pred = FALSE, verbose = FALSE) |
| 130 | +
|
| 131 | +print(grid) |
| 132 | +``` |
| 133 | + |
| 134 | +### Running the Tuning Process |
| 135 | + |
| 136 | +Now, we run `tune::tune_grid` to perform the actual tuning. For demonstration purposes, we'll use a small number of resamples and a simple dataset. |
| 137 | + |
| 138 | +```{r run_tuning} |
| 139 | +# Run tuning |
| 140 | +tune_res <- tune::tune_grid( |
| 141 | + tune_wf, |
| 142 | + resamples = rsample::vfold_cv(iris, v = 2), |
| 143 | + grid = grid, |
| 144 | + control = control |
| 145 | +) |
| 146 | +
|
| 147 | +print(tune_res) |
| 148 | +``` |
| 149 | + |
| 150 | +### Visualizing Results with `autoplot()` |
| 151 | + |
| 152 | +With the tuning complete, we can now use `ggplot2::autoplot()` to visualize the results. Because we provided unique `id`s, `autoplot()` can correctly generate separate plots for each tuned parameter. |
| 153 | + |
| 154 | +```{r autoplot_results, fig.width=7, fig.height=5} |
| 155 | +# Assert that autoplot works without error |
| 156 | +ggplot2::autoplot(tune_res) |
| 157 | +``` |
| 158 | + |
| 159 | +As you can see, `autoplot()` successfully generates a plot showing the performance across the different values for `dense_layer_one_units` and `dense_layer_two_units` independently. |
| 160 | + |
| 161 | +## Why Unique `id`s are Necessary |
| 162 | + |
| 163 | +Internally, `kerasnip` maps arguments like `units` from your `layer_blocks` functions to appropriate `dials` parameter objects (e.g., `dials::hidden_units()`). When multiple such arguments exist, they all point to the *same type* of `dials` parameter. |
| 164 | + |
| 165 | +The `id` argument in `tune()` serves as a unique identifier that `tune` and `ggplot2::autoplot()` use to distinguish between different instances of these parameter types. Without it, `autoplot()` would see multiple parameters of type `hidden_units` and wouldn't know how to plot them separately, leading to errors or combining them incorrectly. |
| 166 | + |
| 167 | +## Best Practices |
| 168 | + |
| 169 | +* **Always use unique `id`s:** When tuning multiple parameters that are conceptually similar (e.g., `units` in different layers, `rate` in different dropout layers), always provide a descriptive and unique `id` to the `tune()` function. |
| 170 | +* **Descriptive `id`s:** Choose `id`s that clearly indicate which part of the model the parameter belongs to (e.g., `dense_layer_one_units`, `conv_filter_size`). This improves readability and understanding of your tuning results. |
| 171 | + |
| 172 | +By following this practice, you ensure that your `kerasnip` models are robustly tunable and that their results can be clearly visualized using the `tidymodels` ecosystem. |
| 173 | + |
| 174 | +```{r cleanup, include=FALSE} |
| 175 | +suppressMessages(remove_keras_spec(model_name)) |
| 176 | +``` |
0 commit comments