-
Notifications
You must be signed in to change notification settings - Fork 26
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
* add README for test Signed-off-by: Ping Yu <yuping@pingcap.com> * add old README back Signed-off-by: Ping Yu <yuping@pingcap.com> * fix fmt Signed-off-by: Ping Yu <yuping@pingcap.com> * address comment Signed-off-by: Ping Yu <yuping@pingcap.com> Signed-off-by: Ping Yu <yuping@pingcap.com>
- Loading branch information
Showing
4 changed files
with
160 additions
and
54 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| ## Preparations | ||
|
|
||
| ### Run integration tests locally | ||
|
|
||
| Assume that you are in the root directory of source code (`github.com/tikv/migration/cdc`). | ||
|
|
||
| 1. The following executables must be copied or generated or linked into these locations | ||
|
|
||
| * `scripts/bin/tidb-server` # version >= 6.2.0 | ||
| * `scripts/bin/tikv-server` # version >= 6.2.0 | ||
| * `scripts/bin/pd-server` # version >= 6.2.0 | ||
| * `scripts/bin/pd-ctl` # version >= 6.2.0 | ||
| * [scripts/bin/go-ycsb](https://github.com/pingcap/go-ycsb) | ||
| * [scripts/bin/etcdctl](https://github.com/etcd-io/etcd/tree/master/etcdctl) | ||
| * [scripts/bin/jq](https://stedolan.github.io/jq/) | ||
|
|
||
| > If you are running tests on Linux x86-64, you can run `make prepare_test_binaries` to get all necessary binaries. | ||
| > | ||
| > If you are running tests on MacOS, tidb related binaries can be downloaded from tiup mirrors, such as https://tiup-mirrors.pingcap.com/tidb-v6.2.0-darwin-amd64.tar.gz. | ||
| 2. The user used to execute the tests must have permission to create the folder /tmp/tikv_cdc_test. All test artifacts | ||
| will be written into this folder. | ||
|
|
||
| ### Run integration tests in docker | ||
|
|
||
| The following programs must be installed: | ||
|
|
||
| * [docker](https://docs.docker.com/get-docker/) | ||
| * [docker-compose](https://docs.docker.com/compose/install/) | ||
|
|
||
| We recommend that you provide docker with at least 6+ cores and 8G+ memory. Of course, the more resources, the better. | ||
|
|
||
| ## Running | ||
|
|
||
| ### Unit Test | ||
|
|
||
| 1. Unit test does not need any dependencies, just running `make unit_test` in root dir of source code, or `cd` into | ||
| directory of a test case and run single case via `GO111MODULE=on go test -check.f TestXXX`. | ||
|
|
||
| ### Integration Test | ||
|
|
||
| #### Run integration tests locally | ||
|
|
||
| 1. Run `make integration_test` to execute the integration tests. This command will | ||
|
|
||
| 1. Download all required executables. | ||
| 2. Check that all required executables exist. | ||
| 3. Generate `tikv-cdc` binaries for integration test. | ||
| 4. Execute `tests/integration_tests/run.sh` | ||
|
|
||
| > If want to run one integration test case only, just pass the CASE parameter, e.g. `make integration_test CASE=autorandom`. | ||
| > | ||
| > There are some environment variables that you can set by yourself, see [test_prepare](./integration_tests/_utils/test_prepare). | ||
| #### Run integration tests in docker | ||
|
|
||
| 1. Run `tests/up.sh`. This script will setup a container with some tools ready, and run `/bin/bash` interactively in the container. | ||
|
|
||
| 2. Run `make integration_test` or `make integration_test CASE=[test name]` to execute the integration tests. | ||
|
|
||
| > **Warning:** | ||
| > These scripts and files may not work under the arm architecture, | ||
| > and we have not tested against it. | ||
| Some useful tips: | ||
|
|
||
| - You can specify multiple tests to run in CASE, for example: `CASE="cli cli_tls"`. You can even | ||
| use `CASE="*"` to indicate that you are running all tests。 | ||
|
|
||
| ## Writing new tests | ||
|
|
||
| 1. Write new integration tests as shell scripts in `tests/integration_tests/TEST_NAME/run.sh`. The script should | ||
| exit with a nonzero error code on failure. | ||
|
|
||
| 2. Add TEST_NAME to existing group in [run_group.sh](./integration_tests/run_group.sh), or add a new group for it. | ||
|
|
||
| 3. If you add a new group, the name of the new group must be added to [CI](https://github.com/PingCAP-QE/ci/blob/main/pipelines/tikv/migration/latest/pull_integration_test.groovy). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -eo pipefail | ||
|
|
||
| CUR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) | ||
|
|
||
| group=$1 | ||
|
|
||
| # Define groups | ||
| # Note: If new group is added, the group name must also be added to CI | ||
| # https://github.com/PingCAP-QE/ci/blob/main/pipelines/tikv/migration/latest/pull_integration_test.groovy | ||
| declare -A groups | ||
| groups=( | ||
| ["G00"]='autorandom' | ||
| ["G01"]='capture_session_done_during_task cdc_hang_on' | ||
| ["G02"]='changefeed_auto_stop changefeed_error changefeed_fast_fail' | ||
| ["G03"]='changefeed_finish changefeed_pause_resume changefeed_reconstruct' | ||
| ["G04"]='cli cli_tls http_api http_proxies' | ||
| ["G05"]='disk_full flow_control' | ||
| ["G06"]='gc_safepoint kill_owner' | ||
| ["G07"]='kv_client_stream_reconnect multi_capture' | ||
| ["G08"]='processor_err_chan processor_panic' | ||
| ["G09"]='processor_resolved_ts_fallback processor_stop_delay' | ||
| ["G10"]='sink_hang sink_tls' | ||
| ["G11"]='sorter stop_downstream' | ||
| ["G12"]='availability' # heavy test case | ||
| ) | ||
|
|
||
| # Get other cases not in groups, to avoid missing any case | ||
| others=() | ||
| for script in "$CUR"/*/run.sh; do | ||
| test_name="$(basename "$(dirname "$script")")" | ||
| # shellcheck disable=SC2076 | ||
| if [[ ! " ${groups[*]} " =~ " ${test_name} " ]]; then | ||
| others=("${others[@]} ${test_name}") | ||
| fi | ||
| done | ||
|
|
||
| # Get test names | ||
| test_names="" | ||
| # shellcheck disable=SC2076 | ||
| if [[ "$group" == "others" ]]; then | ||
| test_names="${others[*]}" | ||
| elif [[ " ${!groups[*]} " =~ " ${group} " ]]; then | ||
| test_names="${groups[${group}]}" | ||
| else | ||
| echo "Error: invalid group name: ${group}" | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Run test cases | ||
| if [[ -n $test_names ]]; then | ||
| echo "Run cases: ${test_names}" | ||
| "${CUR}"/run.sh "${test_names}" | ||
| fi |