Skip to content

Add ability to run tests in parallel [PULP-753] #1241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
gerrod3 wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Add ability to run tests in parallel [PULP-753] #1241

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,5 +115,5 @@ jobs:
PULP_ENABLED_PLUGINS: "${{ matrix.pulp_enabled_plugins }}"
OAS_VERSION: "${{ matrix.oas_version }}"
run: |
.ci/run_container.sh make livetest
.ci/run_container.sh make paralleltest
...
3 changes: 3 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ test: | tests/cli.toml
livetest: | tests/cli.toml
python3 -m pytest -v tests pulp-glue/tests -m live

paralleltest: | tests/cli.toml
python3 -m pytest -v tests pulp-glue/tests -m live -n 8

unittest:
python3 -m pytest -v tests pulp-glue/tests cookiecutter/pulp_filter_extension.py -m "not live"

Expand Down
2 changes: 1 addition & 1 deletion cookiecutter/ci/{{ cookiecutter.__project_name }}/.github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,5 +87,5 @@ jobs:
OAS_VERSION: "${{ matrix.oas_version }}"
{%- endraw %}
run: |
.ci/run_container.sh make {% if cookiecutter.unittests %}live{% endif %}test
.ci/run_container.sh make {% if cookiecutter.unittests %}parallel{% endif %}test
...
3 changes: 3 additions & 0 deletions cookiecutter/ci/{{ cookiecutter.__project_name }}/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,9 @@ test: | tests/cli.toml
livetest: | tests/cli.toml
python3 -m pytest -v tests {%- if cookiecutter.glue %} pulp-glue{{ cookiecutter.__app_label_suffix }}/tests {%- endif %} -m live

paralleltest: | tests/cli.toml
python3 -m pytest -v tests {%- if cookiecutter.glue %} pulp-glue{{ cookiecutter.__app_label_suffix }}/tests {%- endif %} -m live -n 8

unittest:
python3 -m pytest -v tests {%- if cookiecutter.glue %} pulp-glue{{ cookiecutter.__app_label_suffix }}/tests {%- endif %}
{%- if cookiecutter.__app_label_suffix == "" %} cookiecutter/pulp_filter_extension.py {%- endif %} -m "not live"
Expand Down
13 changes: 10 additions & 3 deletions pulpcore/cli/container/distribution.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
from pulp_glue.common.context import (
EntityDefinition,
EntityFieldDefinition,
PluginRequirement,
PulpEntityContext,
PulpRepositoryContext,
)
Expand Down Expand Up @@ -114,6 +115,10 @@ def update(

distribution: EntityDefinition = distribution_ctx.entity
body: EntityDefinition = {}
non_blocking = not (
base_path is not None
or distribution_ctx.pulp_ctx.has_plugin(PluginRequirement("core", specifier="<3.24.0"))
)

if private is not None:
body["private"] = private
Expand All @@ -134,16 +139,18 @@ def update(
repository = t.cast(PulpEntityContext, repository)
if version is not None:
if distribution["repository"]:
distribution_ctx.update(body={"repository": ""}, non_blocking=True)
distribution_ctx.update(body={"repository": ""}, non_blocking=non_blocking)
body["repository_version"] = f"{repository.pulp_href}versions/{version}/"
else:
if distribution["repository_version"]:
distribution_ctx.update(body={"repository_version": ""}, non_blocking=True)
distribution_ctx.update(
body={"repository_version": ""}, non_blocking=non_blocking
)
body["repository"] = repository.pulp_href
elif version is not None:
# keep current repository, change version
if distribution["repository"]:
distribution_ctx.update(body={"repository": ""}, non_blocking=True)
distribution_ctx.update(body={"repository": ""}, non_blocking=non_blocking)
body["repository_version"] = f'{distribution["repository"]}versions/{version}/'
elif distribution["repository_version"]:
# 'dummy' vars are to get us around a mypy/1.2 complaint about '_'
Expand Down
4 changes: 4 additions & 0 deletions pulpcore/cli/core/task.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,10 @@ def cancel(
states.append("running")
if states:
tasks = task_ctx.list(limit=1 << 64, offset=0, parameters={"state__in": states})
if pulp_ctx.api._dry_run:
raise click.ClickException(
_("Trying to cancel {} tasks in safe-mode, aborting").format(len(tasks))
)
Comment on lines +156 to +159
Copy link
Member

@mdellweg mdellweg Sep 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 Is this changing the behaviour or just preventing an Exception further down in the stack to trigger?

Copy link
Contributor Author

@gerrod3 gerrod3 Sep 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's changing the behavior to be more consistent. Before if you ran this command in dry mode it would only give the abort message upon the first task cancellation, meaning it would only error if there was running/waiting tasks. Now it will always give an error (with the amount of available tasks to cancel) when ran in dry mode. Basically the test_tasks check of this command was really inconsistent when running in parallel.

for task in tasks:
task_ctx.cancel(task["pulp_href"], background=True)
for task in tasks:
Expand Down
1 change: 1 addition & 0 deletions test_requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Test requirements
pytest>=7.0.0,<8.5
pytest-xdist
pytest-subtests>=0.12.0,<0.15
python-gnupg>=0.5.0,<0.6
trustme>=1.1.0,<1.3
Expand Down
1 change: 1 addition & 0 deletions tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ def pulp_cli_vars(pulp_cli_vars: t.Dict[str, str]) -> t.Dict[str, str]:
result.update(
{
"FILE_REMOTE_URL": urljoin(PULP_FIXTURES_URL, "/file/PULP_MANIFEST"),
"FILE_REMOTE2_URL": urljoin(PULP_FIXTURES_URL, "/file2/PULP_MANIFEST"),
"FILE_LARGE_REMOTE_URL": urljoin(PULP_FIXTURES_URL, "/file-perf/PULP_MANIFEST"),
"CONTAINER_REMOTE_URL": "https://quay.io",
"CONTAINER_IMAGE": "libpod/alpine",
Expand Down
37 changes: 18 additions & 19 deletions tests/scripts/pulp_ansible/test_content.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,35 +7,34 @@ set -eu
pulp debug has-plugin --name "ansible" || exit 23

cleanup() {
pulp ansible repository destroy --name "cli_test_ansible_repository" || true
pulp ansible repository destroy --name "cli_test_ansible_repository_verify" || true
pulp ansible repository destroy --name "cli_test_ansible_repository_upload" || true
pulp ansible repository destroy --name "cli_test_ansible_content_repository" || true
pulp ansible repository destroy --name "cli_test_ansible_content_repository_verify" || true
pulp ansible repository destroy --name "cli_test_ansible_content_repository_upload" || true
}
trap cleanup EXIT

pulp orphan cleanup --protection-time 0

if pulp debug has-plugin --name "ansible" --specifier ">=0.15.0"
then
gpg --output pulp_pubkey.key --armor --export "pulp-fixture-signing-key"
expect_succ pulp ansible repository create --name "cli_test_ansible_repository_verify" --gpgkey @pulp_pubkey.key
expect_succ pulp ansible repository create --name "cli_test_ansible_content_repository_verify" --gpgkey @pulp_pubkey.key
else
expect_succ pulp ansible repository create --name "cli_test_ansible_repository_verify"
expect_succ pulp ansible repository create --name "cli_test_ansible_content_repository_verify"
fi

# Test ansible collection-version upload
wget "https://galaxy.ansible.com/download/ansible-posix-1.3.0.tar.gz"
sha256=$(sha256sum ansible-posix-1.3.0.tar.gz | cut -d' ' -f1)

expect_succ pulp ansible repository create --name "cli_test_ansible_repository_upload"
expect_succ pulp ansible content upload --file "ansible-posix-1.3.0.tar.gz" --repository "cli_test_ansible_repository_upload"
expect_succ pulp ansible repository create --name "cli_test_ansible_content_repository_upload"
expect_succ pulp ansible content upload --file "ansible-posix-1.3.0.tar.gz" --repository "cli_test_ansible_content_repository_upload"
expect_succ pulp artifact list --sha256 "$sha256"
test "$(echo "$OUTPUT" | jq -r length)" -eq "1"
expect_succ pulp ansible content list --name "posix" --namespace "ansible" --version "1.3.0"
test "$(echo "$OUTPUT" | jq -r length)" -eq "1"
content_href="$(echo "$OUTPUT" | jq -r .[0].pulp_href)"
expect_succ pulp ansible content show --href "$content_href"
expect_succ pulp ansible repository content list --repository "cli_test_ansible_repository_upload" --version 1
expect_succ pulp ansible repository content list --repository "cli_test_ansible_content_repository_upload" --version 1
test "$(echo "$OUTPUT" | jq -r length)" -eq "1"

# Test ansible role upload
Expand All @@ -57,29 +56,29 @@ then
tar --extract --file="ansible-posix-1.3.0.tar.gz" "MANIFEST.json"
collection_path="$(realpath 'MANIFEST.json')"
signature_path="$("$(dirname "$(dirname "$(dirname "$(realpath "$0")")")")"/assets/sign_detached.sh "$collection_path" | jq -r .signature)"
expect_succ pulp ansible content --type "signature" upload --file "$signature_path" --collection "$content_href" --repository "cli_test_ansible_repository_verify"
expect_succ pulp ansible content --type "signature" upload --file "$signature_path" --collection "$content_href" --repository "cli_test_ansible_content_repository_verify"
expect_succ pulp ansible content --type "signature" list --collection "$content_href" --pubkey-fingerprint "0C1A894EBB86AFAE218424CADDEF3019C2D4A8CF"
test "$(echo "$OUTPUT" | jq -r length)" -eq "1"
content3_href="$(echo "$OUTPUT" | jq -r .[0].pulp_href)"
expect_succ pulp ansible content --type "signature" show --href "$content3_href"
fi

# New content commands
expect_succ pulp ansible repository create --name "cli_test_ansible_repository"
expect_succ pulp ansible repository content add --repository "cli_test_ansible_repository" --name "posix" --namespace "ansible" --version "1.3.0"
expect_succ pulp ansible repository content list --repository "cli_test_ansible_repository" --version 1
expect_succ pulp ansible repository create --name "cli_test_ansible_content_repository"
expect_succ pulp ansible repository content add --repository "cli_test_ansible_content_repository" --name "posix" --namespace "ansible" --version "1.3.0"
expect_succ pulp ansible repository content list --repository "cli_test_ansible_content_repository" --version 1
test "$(echo "$OUTPUT" | jq -r length)" -eq "1"
expect_succ pulp ansible repository content --type "role" add --repository "cli_test_ansible_repository" --name "kubernetes-modules" --namespace "ansible" --version "0.0.1"
expect_succ pulp ansible repository content --type "role" list --repository "cli_test_ansible_repository" --version 2
expect_succ pulp ansible repository content --type "role" add --repository "cli_test_ansible_content_repository" --name "kubernetes-modules" --namespace "ansible" --version "0.0.1"
expect_succ pulp ansible repository content --type "role" list --repository "cli_test_ansible_content_repository" --version 2
test "$(echo "$OUTPUT" | jq -r length)" -eq "1"

if pulp debug has-plugin --name "core" --specifier ">=3.11.0"
then
expect_succ pulp ansible repository content list --repository "cli_test_ansible_repository" --version 2 --all-types
expect_succ pulp ansible repository content list --repository "cli_test_ansible_content_repository" --version 2 --all-types
test "$(echo "$OUTPUT" | jq -r length)" -eq "2"
fi

expect_succ pulp ansible repository content remove --repository "cli_test_ansible_repository" --href "$content_href"
expect_succ pulp ansible repository content remove --repository "cli_test_ansible_repository" --href "$content2_href"
expect_succ pulp ansible repository content list --repository "cli_test_ansible_repository"
expect_succ pulp ansible repository content remove --repository "cli_test_ansible_content_repository" --href "$content_href"
expect_succ pulp ansible repository content remove --repository "cli_test_ansible_content_repository" --href "$content2_href"
expect_succ pulp ansible repository content list --repository "cli_test_ansible_content_repository"
test "$(echo "$OUTPUT" | jq -r length)" -eq "0"
8 changes: 4 additions & 4 deletions tests/scripts/pulp_ansible/test_distribution.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,21 @@ set -eu
pulp debug has-plugin --name "ansible" || exit 23

cleanup() {
pulp ansible repository destroy --name "cli_test_ansible_repository" || true
pulp ansible repository destroy --name "cli_test_ansible_distribution_repository" || true
pulp ansible distribution destroy --name "cli_test_ansible_distro" || true
pulp ansible distribution destroy --name "cli_test_ansible_ver_distro" || true
}
trap cleanup EXIT

expect_succ pulp ansible repository create --name "cli_test_ansible_repository"
expect_succ pulp ansible repository create --name "cli_test_ansible_distribution_repository"

expect_succ pulp ansible distribution create --name "cli_test_ansible_distro" \
--base-path "cli_test_ansible_distro" \
--repository "cli_test_ansible_repository"
--repository "cli_test_ansible_distribution_repository"

expect_succ pulp ansible distribution create --name "cli_test_ansible_ver_distro" \
--base-path "cli_test_ansible_ver_distro" \
--repository "cli_test_ansible_repository" \
--repository "cli_test_ansible_distribution_repository" \
--version "0"

expect_succ pulp ansible distribution list
Expand Down
15 changes: 7 additions & 8 deletions tests/scripts/pulp_ansible/test_sign.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,25 +8,24 @@ pulp debug has-plugin --name "ansible" --specifier ">=0.12.0" || exit 23
[ "$(pulp signing-service list --name "sign_ansible" | jq 'length')" = "1" ] || exit 23

cleanup() {
pulp ansible remote -t "collection" destroy --name "cli_test_ansible_collection_remote" || true
pulp ansible repository destroy --name "cli_test_ansible_repository" || true
pulp orphan cleanup || true
pulp ansible remote -t "collection" destroy --name "cli_test_ansible_sign_remote" || true
pulp ansible repository destroy --name "cli_test_ansible_sign_repository" || true
}
trap cleanup EXIT

# Prepare
expect_succ pulp ansible remote -t "collection" create --name "cli_test_ansible_collection_remote" \
expect_succ pulp ansible remote -t "collection" create --name "cli_test_ansible_sign_remote" \
--url "$ANSIBLE_COLLECTION_REMOTE_URL" --requirements "collections:
- robertdebock.ansible_development_environment"
expect_succ pulp ansible repository create --name "cli_test_ansible_repository"
expect_succ pulp ansible repository create --name "cli_test_ansible_sign_repository"
HREF="$(echo "$OUTPUT" | jq -r '.pulp_href')"
expect_succ pulp ansible repository sync --repository "cli_test_ansible_repository" --remote "cli_test_ansible_collection_remote"
expect_succ pulp ansible repository sync --repository "cli_test_ansible_sign_repository" --remote "cli_test_ansible_sign_remote"

# Test sign
expect_succ pulp ansible repository sign --repository "cli_test_ansible_repository" --signing-service "sign_ansible"
expect_succ pulp ansible repository sign --repository "cli_test_ansible_sign_repository" --signing-service "sign_ansible"

# Verify sign
expect_succ pulp ansible repository version list --repository "$HREF"
test "$(echo "$OUTPUT" | jq -r length)" -eq 3
expect_succ pulp ansible repository version show --repository "cli_test_ansible_repository" --version 2
expect_succ pulp ansible repository version show --repository "cli_test_ansible_sign_repository" --version 2
test "$(echo "$OUTPUT" | jq -r '.content_summary.present."ansible.collection_signature".count')" -gt 0
37 changes: 18 additions & 19 deletions tests/scripts/pulp_ansible/test_sync.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,57 +7,56 @@ set -eu
pulp debug has-plugin --name "ansible" || exit 23

cleanup() {
pulp ansible remote -t "role" destroy --name "cli_test_ansible_remote" || true
pulp ansible remote -t "collection" destroy --name "cli_test_ansible_collection_remote" || true
pulp ansible repository destroy --name "cli_test_ansible_repository" || true
pulp orphan cleanup || true
pulp ansible remote -t "role" destroy --name "cli_test_ansible_sync_role_remote" || true
pulp ansible remote -t "collection" destroy --name "cli_test_ansible_sync_remote" || true
pulp ansible repository destroy --name "cli_test_ansible_sync_repository" || true
}
trap cleanup EXIT

cleanup

# Prepare
expect_succ pulp ansible remote -t "role" create --name "cli_test_ansible_remote" --url "$ANSIBLE_ROLE_REMOTE_URL"
expect_succ pulp ansible remote -t "collection" create --name "cli_test_ansible_collection_remote" \
expect_succ pulp ansible remote -t "role" create --name "cli_test_ansible_sync_role_remote" --url "$ANSIBLE_ROLE_REMOTE_URL"
expect_succ pulp ansible remote -t "collection" create --name "cli_test_ansible_sync_remote" \
--url "$ANSIBLE_COLLECTION_REMOTE_URL" --requirements "collections:
- robertdebock.ansible_development_environment"
expect_succ pulp ansible repository create --name "cli_test_ansible_repository"
expect_succ pulp ansible repository create --name "cli_test_ansible_sync_repository"

# Test without remote (should fail)
expect_fail pulp ansible repository sync --repository "cli_test_ansible_repository"
expect_fail pulp ansible repository sync --repository "cli_test_ansible_sync_repository"
# Test with remote
expect_succ pulp ansible repository sync --repository "cli_test_ansible_repository" --remote "role:cli_test_ansible_remote"
expect_succ pulp ansible repository sync --repository "cli_test_ansible_sync_repository" --remote "role:cli_test_ansible_sync_role_remote"

# Preconfigure remote
expect_succ pulp ansible repository update --repository "cli_test_ansible_repository" --remote "role:cli_test_ansible_remote"
expect_succ pulp ansible repository update --repository "cli_test_ansible_sync_repository" --remote "role:cli_test_ansible_sync_role_remote"

# Test with remote
expect_succ pulp ansible repository sync --repository "cli_test_ansible_repository"
expect_succ pulp ansible repository sync --repository "cli_test_ansible_sync_repository"
# Test without remote
expect_succ pulp ansible repository sync --repository "cli_test_ansible_repository" --remote "role:cli_test_ansible_remote"
expect_succ pulp ansible repository sync --repository "cli_test_ansible_sync_repository" --remote "role:cli_test_ansible_sync_role_remote"

# Verify sync
expect_succ pulp ansible repository version list --repository "cli_test_ansible_repository"
expect_succ pulp ansible repository version list --repository "cli_test_ansible_sync_repository"
test "$(echo "$OUTPUT" | jq -r length)" -eq 2
expect_succ pulp ansible repository version show --repository "cli_test_ansible_repository" --version 1
expect_succ pulp ansible repository version show --repository "cli_test_ansible_sync_repository" --version 1
test "$(echo "$OUTPUT" | jq -r '.content_summary.present."ansible.role".count')" -gt 0
expect_succ pulp ansible content --type role list
test "$(echo "$OUTPUT" | jq -r length)" -gt 0

# Test repair the version
expect_succ pulp ansible repository version repair --repository "cli_test_ansible_repository" --version 1
expect_succ pulp ansible repository version repair --repository "cli_test_ansible_sync_repository" --version 1
test "$(echo "$OUTPUT" | jq -r '.state')" = "completed"

# Delete version again
expect_succ pulp ansible repository version destroy --repository "cli_test_ansible_repository" --version 1
expect_succ pulp ansible repository version destroy --repository "cli_test_ansible_sync_repository" --version 1

# Test with collection remote
expect_succ pulp ansible repository sync --name "cli_test_ansible_repository" --remote "cli_test_ansible_collection_remote"
expect_succ pulp ansible repository sync --name "cli_test_ansible_sync_repository" --remote "cli_test_ansible_sync_remote"

# Verify sync
expect_succ pulp ansible repository version list --repository "cli_test_ansible_repository"
expect_succ pulp ansible repository version list --repository "cli_test_ansible_sync_repository"
test "$(echo "$OUTPUT" | jq -r length)" -eq 2
expect_succ pulp ansible repository version show --repository "cli_test_ansible_repository" --version 2
expect_succ pulp ansible repository version show --repository "cli_test_ansible_sync_repository" --version 2
test "$(echo "$OUTPUT" | jq -r '.content_summary.present."ansible.collection_version".count')" -gt 0
expect_succ pulp ansible content list
test "$(echo "$OUTPUT" | jq -r length)" -gt 0
Loading
Loading