Skip to content

Commit

Permalink
update docs for Command Usage and Testing
Browse files Browse the repository at this point in the history
  • Loading branch information
kairu-ms committed Jul 21, 2023
1 parent bcbd992 commit 692f746
Showing 1 changed file with 30 additions and 27 deletions.
57 changes: 30 additions & 27 deletions docs/pages/usage/command_usage_testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,44 +5,47 @@ permalink: /pages/usage/command-usage-testing/
weight: 103
---

## Overview
## Command Usage

Azure CLI provides two types of tests we can use: [unit tests](https://en.wikipedia.org/wiki/Unit_testing) and [integration tests](https://en.wikipedia.org/wiki/Integration_testing).
### Before

For unit tests, we support unit tests written in the forms standard [unittest](https://docs.python.org/3/library/unittest.html).
1. Make sure you have login by `az login`.
2. Make sure you have set the subscription by `az account set -s {subscription ID for test}`.
3. You can try to create a empty resource group by `az group create --name {your resource group name} --location westus` to check if you have the permission to the cloud.
4. If your commands are in extensions, make sure you have loaded this extension by `azdev extension add {your extension name}`.

For integration tests, we provide the `ScenarioTest` and `LiveScenarioTest` classes to support replayable tests via [VCR.py](https://vcrpy.readthedocs.io/en/latest/).
### Run your commands

Details about these two types of testing, env preparation, test policies and issue troubleshooting, please refer to the [doc](https://github.com/Azure/azure-cli/blob/dev/doc/authoring_tests.md)
After that, you can directly try the generated command such as `az new-group-name new-command-name --the-arg value`.

After generating code from CodeGenV2, target cmd's test can be added into folder `path/to/your/cloned/azure-cli/target_mod`. Below is the demonstration of tests using azdev and [Pycharm Community](https://www.jetbrains.com/pycharm/download/#section=linux)
### unit test
There are some tips for you to use the generated command:

When first run unit test, use `--live` to access api and record `yaml` file for integration tests.
```
live mode: azdev test test_function_of_your_code --live
non-live mode: azdev test test_function_of_your_code
```
For issues debugging, use `--debug` parameter appended the above cmd.
1. You can add `--help` argument flag to get and verify the help information of the command. Such ass `az new-group-name new-command-name --the-arg value --help`
2. For issue debugging, you can add `--debug` argument flag to print out the debug logs, which will contains all the http requests and responses, to help you triage the issue. If you need feature help from azure-cli team, please always provide this logs for us.
3. The compound arguments generated supports [shorthand syntax](https://github.com/Azure/azure-cli/blob/dev/doc/shorthand_syntax.md) to input values.

For Pycharm Community, set env `Azure_TEST_RUN_LIVE` to be `true` for live mode and `false` for non-live mode.
## Provide tests

![pycharm_env](../../assets/images/pycharm_live.png)
It is required to providing tests for the change. Please reference this [doc](https://github.com/Azure/azure-cli/blob/dev/doc/authoring_tests.md) for details.

### intergration test
Details about these two types of testing, env preparation, test policies and issue troubleshooting, please refer to the [doc](https://github.com/Azure/azure-cli/blob/dev/doc/authoring_tests.md)

Run tests for specific modules
```
azdev test {mod1} {mod2}
```
Re-run the tests that failed the previous run.
After generating code from CodeGenV2, target cmd's test can be added into folder `path/to/your/cloned/azure-cli/target_mod`. Below is the demonstration of tests using azdev and [Pycharm Community](https://www.jetbrains.com/pycharm/download/#section=linux)

### Run tests

The `azdev test` command can help you run tests for your code, please check help message of `azdev test --help`.

When first run tests, use `--live` to make live request to apis and it will help record payloads in `yaml` file for CI playback validation.
```
azdev test --lf
azdev test test_function_of_your_code --live
```
For more details about using `azdev`, please check message of `azdev test --help`

### Cmd usage
For `Pycharm` users, please set env `Azure_TEST_RUN_LIVE` to be `true` for live mode and `false` for non-live mode.

![pycharm_env](../../assets/images/pycharm_live.png)

1. Before using the generated cmd, `az login` and `az account set -s your-subscription-id` should be set properly as noted [here](https://github.com/Azure/aaz-dev-tools#before-using-generated-commands)
2. Run generated cmd `az your-module your-cmd`
3. For issue debugging, please use `az your-module your-cmd --debug` to check the error and fix it. If more help or instruction needed, please leave a message to us with the error info printed in console.
> **Note**
>
> When you recording tests, please make sure the latest change of `dev` branch of **azure-cli** has been synced to your local repo.
> Developers will always encounter this issue when the tests passed locally but failed in CI. This is because the api versions used by commands of **azure-cli** can be changed in `dev` branch. When the tests recorded in your local may uses out of date azure-cli commands, the recording will be failed in CI. That's why we need to sync the latest change of `dev` branch of **azure-cli** before recording tests.

0 comments on commit 692f746

Please sign in to comment.