From 692f7464de79b13e58505e8bad082f9b3a0e969c Mon Sep 17 00:00:00 2001 From: kai ru <69238381+kairu-ms@users.noreply.github.com> Date: Fri, 21 Jul 2023 11:50:57 +0800 Subject: [PATCH] update docs for Command Usage and Testing --- docs/pages/usage/command_usage_testing.md | 57 ++++++++++++----------- 1 file changed, 30 insertions(+), 27 deletions(-) diff --git a/docs/pages/usage/command_usage_testing.md b/docs/pages/usage/command_usage_testing.md index 7c483f3a..eb04d158 100644 --- a/docs/pages/usage/command_usage_testing.md +++ b/docs/pages/usage/command_usage_testing.md @@ -5,44 +5,47 @@ permalink: /pages/usage/command-usage-testing/ weight: 103 --- -## Overview +## Command Usage -Azure CLI provides two types of tests we can use: [unit tests](https://en.wikipedia.org/wiki/Unit_testing) and [integration tests](https://en.wikipedia.org/wiki/Integration_testing). +### Before -For unit tests, we support unit tests written in the forms standard [unittest](https://docs.python.org/3/library/unittest.html). +1. Make sure you have login by `az login`. +2. Make sure you have set the subscription by `az account set -s {subscription ID for test}`. +3. You can try to create a empty resource group by `az group create --name {your resource group name} --location westus` to check if you have the permission to the cloud. +4. If your commands are in extensions, make sure you have loaded this extension by `azdev extension add {your extension name}`. -For integration tests, we provide the `ScenarioTest` and `LiveScenarioTest` classes to support replayable tests via [VCR.py](https://vcrpy.readthedocs.io/en/latest/). +### Run your commands -Details about these two types of testing, env preparation, test policies and issue troubleshooting, please refer to the [doc](https://github.com/Azure/azure-cli/blob/dev/doc/authoring_tests.md) +After that, you can directly try the generated command such as `az new-group-name new-command-name --the-arg value`. -After generating code from CodeGenV2, target cmd's test can be added into folder `path/to/your/cloned/azure-cli/target_mod`. Below is the demonstration of tests using azdev and [Pycharm Community](https://www.jetbrains.com/pycharm/download/#section=linux) -### unit test +There are some tips for you to use the generated command: -When first run unit test, use `--live` to access api and record `yaml` file for integration tests. -``` -live mode: azdev test test_function_of_your_code --live -non-live mode: azdev test test_function_of_your_code -``` -For issues debugging, use `--debug` parameter appended the above cmd. +1. You can add `--help` argument flag to get and verify the help information of the command. Such ass `az new-group-name new-command-name --the-arg value --help` +2. For issue debugging, you can add `--debug` argument flag to print out the debug logs, which will contains all the http requests and responses, to help you triage the issue. If you need feature help from azure-cli team, please always provide this logs for us. +3. The compound arguments generated supports [shorthand syntax](https://github.com/Azure/azure-cli/blob/dev/doc/shorthand_syntax.md) to input values. -For Pycharm Community, set env `Azure_TEST_RUN_LIVE` to be `true` for live mode and `false` for non-live mode. +## Provide tests -![pycharm_env](../../assets/images/pycharm_live.png) +It is required to providing tests for the change. Please reference this [doc](https://github.com/Azure/azure-cli/blob/dev/doc/authoring_tests.md) for details. -### intergration test +Details about these two types of testing, env preparation, test policies and issue troubleshooting, please refer to the [doc](https://github.com/Azure/azure-cli/blob/dev/doc/authoring_tests.md) -Run tests for specific modules -``` -azdev test {mod1} {mod2} -``` -Re-run the tests that failed the previous run. +After generating code from CodeGenV2, target cmd's test can be added into folder `path/to/your/cloned/azure-cli/target_mod`. Below is the demonstration of tests using azdev and [Pycharm Community](https://www.jetbrains.com/pycharm/download/#section=linux) + +### Run tests + +The `azdev test` command can help you run tests for your code, please check help message of `azdev test --help`. + +When first run tests, use `--live` to make live request to apis and it will help record payloads in `yaml` file for CI playback validation. ``` -azdev test --lf +azdev test test_function_of_your_code --live ``` -For more details about using `azdev`, please check message of `azdev test --help` -### Cmd usage +For `Pycharm` users, please set env `Azure_TEST_RUN_LIVE` to be `true` for live mode and `false` for non-live mode. + +![pycharm_env](../../assets/images/pycharm_live.png) -1. Before using the generated cmd, `az login` and `az account set -s your-subscription-id` should be set properly as noted [here](https://github.com/Azure/aaz-dev-tools#before-using-generated-commands) -2. Run generated cmd `az your-module your-cmd` -3. For issue debugging, please use `az your-module your-cmd --debug` to check the error and fix it. If more help or instruction needed, please leave a message to us with the error info printed in console. +> **Note** +> +> When you recording tests, please make sure the latest change of `dev` branch of **azure-cli** has been synced to your local repo. +> Developers will always encounter this issue when the tests passed locally but failed in CI. This is because the api versions used by commands of **azure-cli** can be changed in `dev` branch. When the tests recorded in your local may uses out of date azure-cli commands, the recording will be failed in CI. That's why we need to sync the latest change of `dev` branch of **azure-cli** before recording tests.