Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try testing with Scapy version updated to 2.5.0 #212

Merged
merged 28 commits into from
Feb 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
58fca24
Try testing with Scapy version updated to 2.5.0
jafingerhut Feb 16, 2025
ff3345b
Changes attempting to get build.yml workflow working on Ubuntu 24.04
jafingerhut Feb 16, 2025
f77941f
More changes to attempt to get tests passing on Ubuntu 24.04 ...
jafingerhut Feb 16, 2025
e55f231
Make cmake builds more verbose
jafingerhut Feb 16, 2025
4dc21c7
Make nanomsg install the same as what I use in my p4-guide script
jafingerhut Feb 16, 2025
f4838fc
A little more debug in the nanomsg install
jafingerhut Feb 16, 2025
7762e30
Extra 'pip --verbose list' commands to see where packages are installed
jafingerhut Feb 16, 2025
7602f72
More tweaks
jafingerhut Feb 16, 2025
15ecb61
Do not upgrade scapy to version later than 2.5.0
jafingerhut Feb 16, 2025
7924ffa
Disable --verbose for nanomsg cmake install
jafingerhut Feb 16, 2025
997d4f5
Install scapy version 2.5.0
jafingerhut Feb 16, 2025
df9c770
More debug info
jafingerhut Feb 16, 2025
78a6d6b
More setting of PYTHONPATH in sudo commands where needed
jafingerhut Feb 16, 2025
affca33
More tweaks to try to get all tests to pass
jafingerhut Feb 16, 2025
5755a87
Automate detection of Python major & minor version
jafingerhut Feb 16, 2025
0413aa4
More tweaking
jafingerhut Feb 16, 2025
4ebea59
Remove some debug commands that seem excessive now
jafingerhut Feb 16, 2025
462c175
Try testing on multiple Ubuntu versions
jafingerhut Feb 16, 2025
cbdab40
Also run test on multiple Ubuntu versions
jafingerhut Feb 16, 2025
cdfadb7
Trim versions to just Ubuntu 22.04 and latest
jafingerhut Feb 16, 2025
e73fa13
Add a second version of Scapy to the test matrix
jafingerhut Feb 16, 2025
c557643
Maybe a fix for dangling CI test
jafingerhut Feb 16, 2025
cce5f72
Address review comments
jafingerhut Feb 18, 2025
e6b4fd3
Merge remote-tracking branch 'up/main' into try-scapy-2.5.0
jafingerhut Feb 18, 2025
75bfc78
Try sudo -E for running Python code as root
jafingerhut Feb 18, 2025
89d920c
Try sudo open --preserve-env for CI test scripts
jafingerhut Feb 18, 2025
c138cca
Reverting previous change, which led to failing CI tests
jafingerhut Feb 18, 2025
c8ab50f
Fix typo
jafingerhut Feb 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 22 additions & 10 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,41 +11,53 @@ on:

jobs:
verify-python:
strategy:
matrix:
os: [ubuntu-22.04, ubuntu-24.04]
scapy_version: [2.5.0, 2.6.1]
# Don't abort other runs when one of them fails, to ease debugging.
fail-fast: false

name: Python code verification (src)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call this linting or static analysis. Minor comment.

I updated the CI to require these tests now.

runs-on: ubuntu-22.04
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Install dependencies
run: |
make set-dev
python -m pip install scapy==2.4.5
python -m pip install scapy==${{ matrix.scapy_version }}
- name: Verify code (python w/black)
run: |
make format-check
- name: Run pytests (utests/tests)
run: |
make test
test:
runs-on: ubuntu-22.04
strategy:
matrix:
os: [ubuntu-22.04, ubuntu-24.04]
scapy_version: [2.5.0, 2.6.1]
# Don't abort other runs when one of them fails, to ease debugging.
fail-fast: false

name: run ptf unit tests
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4

- name: Dependencies
run: |
sudo apt-get install cmake libffi-dev ethtool python3-dev
# Install the latest pip version (not python3-pip) as older versions
# seem to have issues when installing from source.
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python3 get-pip.py
sudo python3 -m pip install --upgrade setuptools nose2 scapy wheel
python3 -m pip install --upgrade setuptools nose2 wheel
python -m pip install scapy==${{ matrix.scapy_version }}

bash CI/install-nanomsg.sh
sudo ldconfig
bash CI/install-nnpy.sh

- name: Install
run: |
sudo python3 -m pip install .
python3 -m pip install .
ptf --version

- name: Before_script
Expand All @@ -54,6 +66,6 @@ jobs:

- name: Script
run: |
pip --verbose list
python3 CI/check-nnpy.py
./CI/run_tests.sh
sudo python3 /usr/local/bin/nose2 utests.tests.test
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI for reviewers - these tests are still run, but I moved the command that runs them into the ./CI/run_tests.sh script, for easier control of additional command line options.

3 changes: 3 additions & 0 deletions CI/install-nanomsg.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
THIS_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
source $THIS_DIR/common.sh

set -x
cmake --version
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the motivation for this output?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debugging CI script behavior.


# nanomsg is very confusing in how it manages SOVERSION vs VERSION, but this
# should be okay... (5.0.0 is the SOVERSION)
check_lib libnanomsg libnanomsg.so.5.0.0
Expand Down
6 changes: 3 additions & 3 deletions CI/install-nnpy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
set -e
git clone https://github.com/nanomsg/nnpy.git
cd nnpy
sudo python3 -m pip install cffi
sudo python3 -m pip install --upgrade cffi
sudo python3 -m pip install .
python3 -m pip install cffi
python3 -m pip install --upgrade cffi
python3 -m pip install .
cd ..
18 changes: 11 additions & 7 deletions CI/run_tests.sh
Original file line number Diff line number Diff line change
@@ -1,23 +1,27 @@
#!/usr/bin/env bash

sudo python ptf_nn/ptf_nn_agent.py \
set -x

sudo -E PATH=${PATH} python3 ptf_nn/ptf_nn_agent.py \
--device-socket 0@tcp://127.0.0.1:10001 -i 0-1@veth0 \
&>/dev/null &
&

sleep 5

sudo python ptf_nn/ptf_nn_agent.py \
sudo -E PATH=${PATH} python3 ptf_nn/ptf_nn_agent.py \
--device-socket 1@tcp://127.0.0.1:10002 -i 1-1@veth3 \
&>/dev/null &
&

sleep 5

sudo python ptf_nn/ptf_nn_test_bridge.py -ifrom veth1 -ito veth2 \
&>/dev/null &
sudo -E PATH=${PATH} python3 ptf_nn/ptf_nn_test_bridge.py -ifrom veth1 -ito veth2 \
&

sleep 5

sudo ptf --test-dir ptf_nn/ptf_nn_test \
sudo -E PATH=${PATH} `which ptf` --test-dir ptf_nn/ptf_nn_test \
--device-socket 0-{0-64}@tcp://127.0.0.1:10001 \
--device-socket 1-{0-64}@tcp://127.0.0.1:10002 \
--platform nn

sudo -E PATH=${PATH} python3 `which nose2` utests.tests.test
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,15 @@ More information about Black, you find at
The following software is required to run PTF:

* Python 3.x
* Scapy 2.4.5 (unless you provide another packet manipulation module)
* Scapy 2.5.0 (unless you provide another packet manipulation module)
* pypcap (optional - VLAN tests will fail without this)
* tcpdump (optional - Scapy will complain if it's missing)

Root/sudo privilege is required on the host, in order to run `ptf`.

The default packet manipulator tool for `ptf` is `Scapy`. To install it use:
```text
pip install scapy==2.4.5
pip install scapy==2.5.0
```

To enable VLAN tests, you need to install `pypcap`:
Expand Down