New cli commands help, info-config, errors, status, results + Documen…

…tation (#139)

docs/reference/cli.md

@@ -0,0 +1,165 @@
+# NePS Command Line Interface
+This section provides a brief overview of the commands available in the NePS CLI.
+
+## **`init` Command**
+
+Generates a default `run_args` YAML configuration file, providing a template that you can customize for your experiments.
+
+**Arguments:**
+
+
+- `-h, --help` (Optional): show this help message and exit
+- `--config-path` (Optional): Optional custom path for generating the configuration file. Default is 'run_config.yaml'.
+- `--template` (Optional): Optional, options between different templates. Required configs(basic) vs all neps configs (complete)
+- `--state-machine` (Optional): If set, creates a NEPS state. Requires an existing config.yaml.
+
+
+**Example Usage:**
+
+```bash
+neps init --config-path custom/path/config.yaml --template complete
+```
+
+
+## **`run` Command**
+
+Executes the optimization based on the provided configuration. This command serves as a CLI wrapper around `neps.run`, effectively mapping each CLI argument to a parameter in `neps.run`. It offers a flexible interface that allows you to override the existing settings specified in the YAML configuration file, facilitating dynamic adjustments for managing your experiments.
+
+**Arguments:**
+
+
+- `-h, --help` (Optional): show this help message and exit
+- `--run-args` (Optional): Path to the YAML configuration file.
+- `--run-pipeline` (Optional): Optional: Provide the path to a Python file and a function name separated by a colon, e.g., 'path/to/module.py:function_name'. If provided, it overrides the run_pipeline setting from the YAML configuration.
+- `--pipeline-space` (Optional): Path to the YAML file defining the search space for the optimization. This can be provided here or defined within the 'run_args' YAML file.
+- `--root-directory` (Optional): The directory to save progress to. This is also used to synchronize multiple calls for parallelization.
+- `--overwrite-working-directory` (Optional): If set, deletes the working directory at the start of the run. This is useful, for example, when debugging a run_pipeline function.
+- `--development-stage-id` (Optional): Identifier for the current development stage, used in multi-stage projects.
+- `--task-id` (Optional): Identifier for the current task, useful in projects with multiple tasks.
+- `--post-run-summary` (Optional): Provide a summary of the results after running.
+- `--no-post-run-summary` (Optional): Do not provide a summary of the results after running.
+- `--max-evaluations-total` (Optional): Total number of evaluations to run.
+- `--max-evaluations-per-run` (Optional): Number of evaluations a specific call should maximally do.
+- `--continue-until-max-evaluation-completed` (Optional): If set, only stop after max-evaluations-total have been completed. This is only relevant in the parallel setting.
+- `--max-cost-total` (Optional): No new evaluations will start when this cost is exceeded. Requires returning a cost
+  in the run_pipeline function.
+- `--ignore-errors` (Optional): If set, ignore errors during the optimization process.
+- `--loss-value-on-error` (Optional): Loss value to assume on error.
+- `--cost-value-on-error` (Optional): Cost value to assume on error.
+- `--searcher` (Optional): String key of searcher algorithm to use for optimization.
+- `--searcher-kwargs` (Optional): Additional keyword arguments as key=value pairs for the searcher.
+
+
+**Example Usage:**
+
+```bash
+neps run --run-args path/to/config.yaml --max-evaluations-total 50
+```
+
+
+## **`status` Command**
+Check the status of the NePS run. This command provides a summary of trials, including pending, evaluating, succeeded, and failed trials. You can filter the trials displayed based on their state.
+
+**Arguments:**
+
+
+- `-h, --help` (Optional): show this help message and exit
+- `--root-directory` (Optional): Optional: The path to your root_directory. If not provided, it will be loaded from run_config.yaml.
+- `--pending` (Optional): Show only pending trials.
+- `--evaluating` (Optional): Show only evaluating trials.
+- `--succeeded` (Optional): Show only succeeded trials.
+
+
+**Example Usage:**
+```bash
+neps status --root-directory path/to/directory --succeeded
+```
+
+
+## **`info-config` Command**
+Provides detailed information about a specific configuration identified by its ID. This includes metadata, configuration values, and trial status.
+
+**Arguments:**
+
+
+- id (Required): The configuration ID to be used.
+
+
+- `-h, --help` (Optional): show this help message and exit
+- `--root-directory` (Optional): Optional: The path to your root_directory. If not provided, it will be loaded from run_config.yaml.
+
+
+**Example Usage:**
+```bash
+neps info-config 42 --root-directory path/to/directory
+```
+
+## **`results` Command**
+Displays the results of the NePS run, listing all incumbent trials in reverse order (most recent first). Optionally,
+you can plot the results to visualize the progression of incumbents over trials.
+
+**Arguments:**
+
+
+- `-h, --help` (Optional): show this help message and exit
+- `--root-directory` (Optional): Optional: The path to your root_directory. If not provided, it will be loaded from run_config.yaml.
+- `--plot` (Optional): Plot the results if set.
+
+
+**Example Usage:**
+
+```bash
+neps results --root-directory path/to/directory --plot
+```
+
+
+
+## **`errors` Command**
+Lists all errors found in the specified NePS run. This is useful for debugging or reviewing failed trials.
+
+**Arguments:**
+
+
+- `-h, --help` (Optional): show this help message and exit
+- `--root-directory` (Optional): Optional: The path to your root_directory. If not provided, it will be loaded from run_config.yaml.
+
+
+**Example Usage:**
+
+```bash
+neps errors --root-directory path/to/directory
+```
+
+
+## **`sample-config` Command**
+
+
+**Arguments:**
+
+
+- `-h, --help` (Optional): show this help message and exit
+- `--root-directory` (Optional): Optional: The path to your root_directory. If not provided, it will be loaded from run_config.yaml.
+
+
+**Example Usage:**
+
+```bash
+neps sample-config --help
+```
+
+
+## **`help` Command**
+Displays help information for the NePS CLI, including a list of available commands and their descriptions.
+
+**Arguments:**
+
+
+- `-h, --help` (Optional): show this help message and exit
+
+
+**Example Usage:**
+
+```bash
+neps help --help
+```
+

docs/reference/declarative_usage.md

@@ -212,15 +212,51 @@ provided by each command using --help.
 
 
 ### **`init` Command**
-Generates a default `run_args` YAML configuration file, providing a template that you can customize for
-your experiments.
+
+Generates a default `run_args` YAML configuration file, providing a template that you can customize for your experiments.
+
+**Options:**
+
+  - `--config-path <path>`: *Optional*. Specify the custom path for generating the configuration file. Defaults to
+  `run_config.yaml` in the current working directory.
+  - `--template [basic|complete]`: *Optional*. Choose between a basic or complete template. The basic template includes only required settings, while the complete template includes all NePS configurations.
+  - `--state-machine`: *Optional*. Creates a NEPS state if set, which requires an existing `run_config.yaml`.
 
 **Example Usage:**
 ```bash
-neps init --config-path custom/path/config.yaml
+neps init --config-path custom/path/config.yaml --template complete
 ```
 
 ### **`run` Command**
+
+Executes the optimization based on the provided configuration. This command serves as a CLI wrapper around `neps.run`, effectively mapping each CLI argument to a parameter in `neps.run`. It offers a flexible interface that allows you to override the existing settings specified in the YAML configuration file, facilitating dynamic adjustments for managing your experiments.
+
+**Options:**
+
+  - `--run-args <path>`: Path to the YAML configuration file containing the run arguments.
+  - `--run-pipeline <path/to/module:function_name>`: *Optional*. Specify the path to the Python module and function to use for running the pipeline. Overrides any settings in the YAML file.
+  - `--pipeline-space <path/to/yaml>`: Path to the YAML file defining the search space for the optimization.
+  - `--root-directory <path>`: *Optional*. Directory for saving progress and synchronizing multiple processes. Defaults to the `root_directory` from `run_config.yaml` if not provided.
+  - `--overwrite-working-directory`: *Optional*. If set, deletes the working directory at the start of the run.
+  - `--development-stage-id <id>`: *Optional*. Identifier for the current development stage, useful for multi-stage projects.
+  - `--task-id <id>`: *Optional*. Identifier for the current task, useful for managing projects with multiple tasks.
+  - `--post-run-summary/--no-post-run-summary`: *Optional*. Provides a summary of the run after execution. Enabled by default.
+  - `--max-evaluations-total <int>`: *Optional*. Specifies the total number of evaluations to run.
+  - `--max-evaluations-per-run <int>`: *Optional*. Number of evaluations to run per call.
+  - `--continue-until-max-evaluation-completed`: *Optional*. If set, ensures the run continues until `max-evaluations-total` has been reached.
+  - `--max-cost-total <float>`: *Optional*. Specifies a cost threshold. No new evaluations will start if this cost is exceeded.
+  - `--ignore-errors`: *Optional*. If set, errors during the optimization will be ignored.
+  - `--loss-value-on-error <float>`: *Optional*. Specifies the loss value to assume in case of an error.
+  - `--cost-value-on-error <float>`: *Optional*. Specifies the cost value to assume in case of an error.
+  - `--searcher <key>`: Specifies the searcher algorithm for optimization.
+  - `--searcher-kwargs <key=value>`: *Optional*. Additional keyword arguments for the searcher.
+
+**Example Usage:**
+```bash
+neps run --run-args path/to/config.yaml --max-evaluations-total 50
+```
+
+### **`status` Command**
 Executes the optimization based on the provided configuration. This command serves as a CLI wrapper around neps.run,
 effectively mapping each CLI argument to a parameter in neps.run. This setup offers a flexible interface that allows
 you to override the existing settings specified in the YAML configuration file, facilitating dynamic adjustments for

mkdocs.yml

@@ -137,6 +137,7 @@ nav:
     - Optimizers: 'reference/optimizers.md'
     - Declarative Usage: 'reference/declarative_usage.md'
     - The Run Function: 'reference/run_pipeline.md'
+    - CLI Usage: 'reference/cli.md'
     - Analysing Runs: 'reference/analyse.md'
   - Examples: "examples/"  # auto-generated
   - API: 'api/' # auto-generated