Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add test script #169

Closed
wants to merge 65 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
47eba8f
docs: add README file into manual-tests directory
PhilipTamb Oct 10, 2023
c91c36c
feat(test): add automated-tests
PhilipTamb Oct 17, 2023
980f98f
style: clean automated-test code
PhilipTamb Oct 17, 2023
0a16a71
feat(test): update after first PR, add requirements.txt
PhilipTamb Oct 18, 2023
a2f8716
feat(test): add getting-started test
PhilipTamb Oct 19, 2023
78a49b6
feat(test): update after Second Review
PhilipTamb Oct 23, 2023
d44041f
feat(test): add automated-test into workflow
PhilipTamb Oct 24, 2023
b3947b1
feat(test): fix zenoh-flow path into test
PhilipTamb Oct 25, 2023
3b290d8
ci: 1t CI workflow test
PhilipTamb Oct 27, 2023
e046371
ci: 2t CI workflow test
PhilipTamb Oct 27, 2023
067860c
ci: 3t CI workflow test
PhilipTamb Oct 27, 2023
29c278d
ci: 4t CI workflow test
PhilipTamb Oct 27, 2023
af3a05a
ci: 5t CI workflow test
PhilipTamb Oct 27, 2023
dfd04fd
ci: 6ts CI workflow test
PhilipTamb Oct 27, 2023
9d27dd4
ci: 7ts CI workflow test
PhilipTamb Oct 27, 2023
cba4d03
ci: 8st CI workflow test
PhilipTamb Oct 28, 2023
60c6201
ci: 9st CI workflow test
PhilipTamb Oct 28, 2023
302ddd9
ci: 10st CI workflow test
PhilipTamb Oct 29, 2023
c8f7dc5
ci: 11st CI workflow test
PhilipTamb Oct 29, 2023
c30b29d
ci: 12st CI workflow test
PhilipTamb Oct 29, 2023
1dfc64b
ci: 12st CI workflow test
PhilipTamb Oct 29, 2023
36199f9
ci: 13st CI workflow test
PhilipTamb Oct 29, 2023
921cebf
ci: 14 CI workflow test
PhilipTamb Oct 30, 2023
0390228
ci: 15 CI workflow test
PhilipTamb Oct 30, 2023
ebddd4e
ci: 16 CI workflow test
PhilipTamb Oct 30, 2023
3498e99
ci: 17 CI workflow test
PhilipTamb Oct 30, 2023
664f26c
ci: 18 CI workflow test
PhilipTamb Oct 30, 2023
c92bee4
ci: 18 CI workflow test
PhilipTamb Oct 30, 2023
ca6ce2b
ci: 19 CI workflow test
PhilipTamb Oct 30, 2023
6b86541
ci: 20 CI workflow test
PhilipTamb Oct 30, 2023
72bbeee
ci: 21 CI workflow test
PhilipTamb Oct 30, 2023
10cc755
ci: 21 CI workflow test
PhilipTamb Oct 31, 2023
5278a3b
ci: 22 CI workflow test
PhilipTamb Oct 31, 2023
a74117f
ci: 23 CI workflow test
PhilipTamb Oct 31, 2023
4582202
ci: 24 CI workflow test, update automated-test.py
PhilipTamb Oct 31, 2023
4e086ff
ci: 24 CI workflow test, update automated-test.py
PhilipTamb Oct 31, 2023
5767b6c
ci: 24 CI workflow test, update automated-test.py
PhilipTamb Oct 31, 2023
e68346e
ci: 25 CI workflow test
PhilipTamb Nov 1, 2023
d05d5fd
ci: 26 CI workflow test, update automated-test.py
PhilipTamb Nov 2, 2023
772c5d5
ci: 27 CI workflow test
PhilipTamb Nov 2, 2023
bb10085
ci: 28 CI workflow test
PhilipTamb Nov 2, 2023
c3ff32e
ci: 28 CI workflow test
PhilipTamb Nov 2, 2023
d8ee7b0
ci: 30 CI workflow test
PhilipTamb Nov 2, 2023
7b655af
ci: 31 CI workflow test
PhilipTamb Nov 2, 2023
300ac39
ci: 32 CI workflow test
PhilipTamb Nov 3, 2023
19bab28
ci: 33 CI workflow test
PhilipTamb Nov 3, 2023
4c98504
ci: 34 CI workflow test
PhilipTamb Nov 3, 2023
c22d71a
ci: 35 CI workflow test
PhilipTamb Nov 3, 2023
e3d4fd9
ci: 36 CI workflow test
PhilipTamb Nov 3, 2023
0d73741
ci: 37 CI workflow test
PhilipTamb Nov 4, 2023
c68eb94
ci: 38 CI workflow test
PhilipTamb Nov 4, 2023
f990ab5
ci: 39 CI workflow test
PhilipTamb Nov 6, 2023
c90d6d8
ci: 40 CI workflow test
PhilipTamb Nov 6, 2023
6f25443
ci: 40 CI workflow test
PhilipTamb Nov 6, 2023
49a3b21
ci: 41 CI workflow test
PhilipTamb Nov 6, 2023
67b1366
ci: 42 CI workflow test
PhilipTamb Nov 6, 2023
2c9d31c
ci: 43 CI workflow test
PhilipTamb Nov 6, 2023
5b16f1b
ci: 43 CI workflow test
PhilipTamb Nov 6, 2023
b44f2b5
ci: 44 CI workflow test
PhilipTamb Nov 6, 2023
83db30d
ci: 45 CI workflow test
PhilipTamb Nov 7, 2023
d92f145
ci: 46 CI workflow test
PhilipTamb Nov 8, 2023
15dcbe1
feat: update automated-test with standalone feature
PhilipTamb Jan 2, 2024
b2e9608
refacto: update zenoh-flow build step
PhilipTamb Jan 2, 2024
cb060ea
refacto: update zenoh-flow build step
PhilipTamb Jan 2, 2024
6a5732e
refacto: update zenoh-flow build step
PhilipTamb Jan 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
249 changes: 233 additions & 16 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,13 @@
name: CI

on:
workflow_dispatch:
push:
branches: [ '**' ]
branches: ["**"]
pull_request:
branches: [ '**' ]
branches: ["**"]
schedule:
- cron: '0 0 * * 1-5'
- cron: "0 0 * * 1-5"

jobs:
checks:
Expand Down Expand Up @@ -68,27 +69,243 @@ jobs:
steps:
- uses: actions/checkout@v2

- name: dump
run: |
echo 'github.workspace === ${{ github.workspace }}'
echo "GITHUB_WORKSPACE === $GITHUB_WORKSPACE"
echo 'runner.workspace === ${{ runner.workspace }}'
echo "RUNNER_WORKSPACE === $RUNNER_WORKSPACE"

- name: Install latest Rust toolchain
uses: actions-rs/toolchain@v1
with:
toolchain: 1.70.0

# questo builda znoh-flow in una temp dir
# - name: Create Zenoh-Flow artifact TEMP_DIR
# run: |
# echo "ZF_RELEASE_TEMP_DIR=$(mktemp -d)" >> $GITHUB_ENV
# echo "${{ env.ZF_RELEASE_TEMP_DIR}} >> ${{ env.ZF_RELEASE_TEMP_DIR}}"

# - name: Build
# uses: actions-rs/cargo@v1
# with:
# command: build
# args: --release --verbose --all-targets --target-dir ZF_RELEASE_TEMP_DIR

- name: Build
uses: actions-rs/cargo@v1
with:
command: build
args: --release --verbose --all-targets
args: --release --verbose --all-targets

- name: Build Zenoh-Flow
run: |
cargo build -p zenoh-flow-standalone-runtime

- name: Run tests
uses: actions-rs/cargo@v1
- name: Build Examples
run: |
cargo build --examples


- name: Find yaml files
uses: Rishabh510/Path-lister-action@master
with:
command: test
args: --release --verbose
env:
ASYNC_STD_THREAD_COUNT: 4
path: "/examples/flows"
type: ".yaml"

- name: Output results
run: |
echo "Found ${{ steps.pl.outputs.path_count }} file(s) with this extension:"
for i in ${{ steps.pl.outputs.paths }}; do
echo $i
done

# - name: Update data-flow yaml
# uses: loveholidays/gitops-action-yaml-updater@v1.0.0
# with:

- name: Run doctests
uses: actions-rs/cargo@v1
# - name: Run tests
# uses: actions-rs/cargo@v1
# with:
# command: test
# args: --release --verbose
# env:
# ASYNC_STD_THREAD_COUNT: 4

# - name: Run doctests
# uses: actions-rs/cargo@v1
# with:
# command: test
# args: --doc
# env:
# ASYNC_STD_THREAD_COUNT: 4

# - name: Check out my other private repo
# uses: actions/checkout@master
# with:
# repository: eclipse-zenoh/zenoh

- name: Create Zenoh TEMP_DIR
run: |
echo "TEMP_DIR=$(mktemp -d)" >> $GITHUB_ENV
echo "${{ env.TEMP_DIR}} >> ${{ env.TEMP_DIR}}"

#./../../zenoh
- name: Clone Repository
run: git clone -b '0.7.2-rc' https://github.com/eclipse-zenoh/zenoh.git ${{ env.TEMP_DIR}}/zenoh

# - name: Rust Version
# run: rustc --version

# - name: Rust toolchain
# run: |
# rustup toolchain install 1.7.0
# rustc --version

# - if: matrix.os == 'ubuntu-latest'
# name: Free Disk Space (Ubuntu)
# uses: jlumbroso/free-disk-space@main
# with:
# # this might remove tools that are actually needed,
# # if set to "true" but frees about 6 GB
# tool-cache: true

- name: Update Rust version
uses: DamianReeves/write-file-action@master
with:
path: ${{env.TEMP_DIR}}/zenoh/rust-toolchain
contents: |
1.70.0
write-mode: overwrite

- name: Build Zenoh
run: |
cd ${{ env.TEMP_DIR}}/zenoh
cargo build --features shared-memory --all-targets

# export PATH="$HOME/.cargo/bin:$PATH"
# rustup override set 1.70.0

# - if: matrix.os == 'macOS-latest'
# name: Free Disk Space (macOS)
# run: |
# top -l 1 -s 0 | grep PhysMem
# sysctl -a | awk '/hw./' && '/mem/'
# top -l 1 -s 0 | grep PhysMem

- if: matrix.os == 'ubuntu-latest'
name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@main
with:
command: test
args: --doc
env:
ASYNC_STD_THREAD_COUNT: 4
# this might remove tools that are actually needed,
# if set to "true" but frees about 6 GB
tool-cache: true

# - name: Print current working dir
# run: |
# cd target
# ls -la $PWD
# ls -la $GITHUB_WORKSPACE

- name: Print into Zenoh
run: |
cd ${{ env.TEMP_DIR}}/zenoh
ls -la $PWD

# - name: Free Disk Space (Ubuntu)
# uses: jlumbroso/free-disk-space@main
# with:
# # this might remove tools that are actually needed,
# # if set to "true" but frees about 6 GB
# tool-cache: true

# - name: Change directory --> from Zenoh-Flow to Home directory
# shell: bash
# run: |
# cd ..
# pwd

# - name: Set Zenoh-Flow plugin path
# run: |
# cd ${{ github.workspace }}
# echo "ZF_PLUGIN_PATH= ${{ github.workspace }}/zenoh-flow-plugin/etc/zenoh-zf-plugin.json" >> $GITHUB_ENV
# echo ${{env.ZF_PLUGIN_PATH}}

# Your action in .github/workflows questo checkout che fa un "Removing previously created refs, to avoid conflicts" potrebbe portare a non avere più le cartelle disponibili
# - name: Checkout repository
# uses:
# actions/checkout@v2
# Repository name with owner. For example, actions/checkout
# Default: ${{ github.repository }}

# - name: Find files matching "*.json"
# uses: jeertmans/filesfinder@latest
# id: ff # Any id, to be used later to reference to files output
# with:
# # Only argument, a single string, to be passed as arguments to ff.
# # See `ff --help` for more help.
# # Default: "*"
# args: -r "zenoh-zf-plugin.json"
# - name: Print files
# run: echo "${{ steps.ff.outputs.files }}"

# - if: matrix.os == 'macOS-latest'
# name: install wget (Ubuntu)
# run: |
# brew install wget

# - if: matrix.os == 'ubuntu-latest'
# name: install wget (Ubuntu)
# run: |
# apt install wget

# - name: Get zenoh-zf-plugin.json
# run: |
# cd ${{ runner.workspace }}
# wget https://github.com/eclipse-zenoh/zenoh-flow/blob/master/zenoh-flow-plugin/etc/zenoh-zf-plugin.json
# ls -la $PWD
# echo "ZF_PLUGIN_PATH= ${{ runner.workspace }}/zenoh-zf-plugin.json" >> $GITHUB_ENV

#echo "ZF_PLUGIN_PATH= ${{ runner.workspace }}/zenoh-flow-plugin/etc/zenoh-zf-plugin.json" >> $GITHUB_ENV
- name: Set path
run: |
echo "ZENOHD_PATH= ${{ env.TEMP_DIR}}/zenoh/target/debug/zenohd" >> $GITHUB_ENV
echo "ZFCTL_PATH= ${{ github.workspace }}/target/release/zfctl" >> $GITHUB_ENV
echo "PY_TEST_PATH= ${{ github.workspace }}/manual-tests/automated-test.py" >> $GITHUB_ENV
echo ${{env.ZENOHD_PATH}}
echo ${{env.ZFCTL_PATH}}
echo ${{env.PY_TEST_PATH}}


#echo "LIB_ZF_PLUGIN_PATH= ${{ github.workspace }}/target/release/" >> $GITHUB_ENV
#echo ${{env.LIB_ZF_PLUGIN_PATH}}
#echo ${{env.ZF_PLUGIN_PATH}}

- name: ls runner.workspace
run: |
cd ${{ runner.workspace }}
ls -la $PWD

- name: ls github.workspace
run: |
cd ${{ github.workspace }}
ls -la $PWD

- name: Install Python package
uses: actions/setup-python@v4
with:
python-version: "3.x"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r manual-tests/requirements.txt

- name: Run Python automated-test.py script
run: |
cd ${{ github.workspace }}
chmod u+x manual-tests/automated-test.py
output=$(python3 automated-test.py --standalone True --zenohd ~/dev/zenoh/target/debug/zenohd )

#
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ vars:
# MacOS -> .dylib
uri: "file://{{ BASE_DIR }}/target/debug/examples/libperiod_miss_detector.dylib"
# If the compilation is in release:
# uri: file:///absolute/path/to/target/release/libperiod_miss_detector.dylib
# uri: file:///absolute/path/to/target/release/libperiod_miss_detector.so

inputs: [in]
outputs: [out]
25 changes: 25 additions & 0 deletions manual-tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Automated Test
## What the script does
In this folder there is a python script to automatically test that zenoh-flow works.

At first the script starts to find the zenoh and zenoh-flow executables and the configuration file paths, this is necessary to build the command to be able to run Zenoh with Zenoh-Flow as plugin.
Furthermore the test makes sure that the zenoh configuration is able to run zenoh-flow as a plugin.

## How does the test work?
This test use the getting-started example to validate the zenoh-flow functionality.

The script test verifies:
- Zenoh with zenoh-flow plugin is up and running.
- The example runs correctly.
- That the number of active Zenoh-Flow nodes is equal to the number of nodes declared into the yaml file.

## How to run
To do this you need to run :
```bash
pip install -r requirements.txt
python3 automated-test.py
```

```bash
usage: automated-test.py -h/--help: to show the script options.
```
Loading
Loading