Skip to content

Move to nproc and Speed Up Ansible Lint #14328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Mab879 wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Move to nproc and Speed Up Ansible Lint #14328

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
872645f
Move to nproc over hard coded 2 to ensure we are fully using CI boxes.
Mab879 Jan 23, 2026
3b38087
Make gate-lint-ansible-roles work in nparallel.
Mab879 Jan 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/gate-lint-ansible-roles.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@ jobs:
run: cmake -DSSG_PRODUCT_RHEL8=ON -DSSG_PRODUCT_RHEL9=ON -DSSG_PRODUCT_RHEL10=ON -G Ninja ..
working-directory: ./build
- name: Build
run: ninja -j2 rhel10-profile-playbooks rhel9-profile-playbooks rhel8-profile-playbooks
run: ninja -j$(nproc) rhel10-profile-playbooks rhel9-profile-playbooks rhel8-profile-playbooks
working-directory: ./build
- name: Build Ansible Roles
run: PYTHONPATH=. python3 utils/ansible_playbook_to_role.py --build-playbooks-dir ./build/ansible/ --dry-run ./build/ansible_roles
# https://github.com/actions/checkout/issues/766
- name: Set git safe directory
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
- name: Lint Ansible Roles
run: ansible-lint -x 204 -x experimental -x command-instead-of-module ./build/ansible_roles/*
run: find ./build/ansible_roles -mindepth 1 -maxdepth 1 -type d | xargs -P $(nproc) -I {} ansible-lint -x 204 -x experimental -x command-instead-of-module {}
Copy link
Collaborator

@vojtapolasek vojtapolasek Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are using many pipes here, would adding something like set -eo pipefail make sense?

Copy link
Member Author

@Mab879 Mab879 Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is only one pipe here and we assume the playbooks exist since the build worked so I don't think it is needed.

Copy link
Collaborator

@vojtapolasek vojtapolasek Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In context of previous discussion it seems it is really not needed.

Copy link
Collaborator

@comps comps Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I would agree with @vojtapolasek here - if the find doesn't find anything and throws an error, ansible-lint would normally scan CWD, but xargs takes over here because nothing was passed, and it simply exits with 0.

$ echo asdasd | xargs -P $(nproc) -I {} false {} ; echo $?
123

$ find /nonexistent | xargs -P $(nproc) -I {} false {} ; echo $?
find: ‘/nonexistent’: No such file or directory
0

Although if we (want to) rely on other steps throwing errors, that's probably fine.

12 changes: 6 additions & 6 deletions .github/workflows/gate.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
- name: Build
run: ./build_product sle12 sle15
- name: Test
run: ctest -j2 --output-on-failure -E unique-stigids
run: ctest -j$(nproc) --output-on-failure -E unique-stigids
working-directory: ./build

validate-suse:
Expand All @@ -55,7 +55,7 @@ jobs:
env:
ADDITIONAL_CMAKE_OPTIONS: "-DSSG_OVAL_SCHEMATRON_VALIDATION_ENABLED=OFF"
- name: Test
run: ctest -j2 --output-on-failure -E unique-stigids
run: ctest -j$(nproc) --output-on-failure -E unique-stigids
working-directory: ./build

validate-debian:
Expand All @@ -81,7 +81,7 @@ jobs:
./build_product debian11 debian12 debian13
- name: Test
working-directory: ./build
run: ctest -j2 --output-on-failure -E unique-stigids
run: ctest -j$(nproc) --output-on-failure -E unique-stigids

validate-ubuntu-22-04:
name: Build, Test on Ubuntu 22.04
Expand All @@ -99,7 +99,7 @@ jobs:
run: |-
./build_product ubuntu2204
- name: Test
run: ctest -j2 --output-on-failure -E unique-stigids
run: ctest -j$(nproc) --output-on-failure -E unique-stigids
working-directory: ./build

validate-ubuntu-24-04:
Expand All @@ -118,7 +118,7 @@ jobs:
run: |-
./build_product ubuntu2404
- name: Test
run: ctest -j2 --output-on-failure -E unique-stigids
run: ctest -j$(nproc) --output-on-failure -E unique-stigids
working-directory: ./build

validate-fedora-rawhide:
Expand All @@ -141,5 +141,5 @@ jobs:
env:
ADDITIONAL_CMAKE_OPTIONS: "-DSSG_OVAL_SCHEMATRON_VALIDATION_ENABLED=OFF"
- name: Test
run: ctest -j2 --output-on-failure -E unique-stigids
run: ctest -j$(nproc) --output-on-failure -E unique-stigids
working-directory: ./build
4 changes: 2 additions & 2 deletions .github/workflows/gate_fedora.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
run: pip install pcre2==0.4.0 -r requirements.txt -r test-requirements.txt
- name: Build
run: |-
./build_product -j2 \
./build_product -j$(nproc) \
al2023 \
alinux2 \
alinux3 \
Expand Down Expand Up @@ -69,7 +69,7 @@ jobs:
unzip /__w/content/content/old_release.zip -d /__w/content/content/old_release
mv /__w/content/content/old_release/*/* /__w/content/content/old_release/
- name: Test
run: ctest -j2 --output-on-failure -E unique-stigids
run: ctest -j$(nproc) --output-on-failure -E unique-stigids
working-directory: ./build
- name: "Set git safe directory, ref: https://github.com/actions/checkout/issues/760"
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/gh-pages.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,13 @@ jobs:
run: cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Debug
working-directory: ./build
- name: Build Guides and Mapping Tables
run: ninja -j2
run: ninja -j$(nproc)
working-directory: ./build
- name: Build Statistics
run: ninja html-stats html-profile-stats -j2
run: ninja html-stats html-profile-stats -j$(nproc)
working-directory: ./build
- name: Render Policies (Using control files)
run: ninja render-policies -j2
run: ninja render-policies -j$(nproc)
working-directory: ./build
- name: Generate Prometheus Metrics
run: utils/controleval_metrics.py prometheus -p fedora ocp4 rhcos4 rhel10 rhel9 rhel8 sle12 sle15 -f ./build/policies_metrics
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/nightly_build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,16 @@ jobs:
run: cmake -G Ninja ..
working-directory: ./build
- name: Build All
run: ninja -j2 all
run: ninja -j$(nproc) all
working-directory: ./build
- name: Build ZIP
run: ninja -j2 zipfile
run: ninja -j$(nproc) zipfile
working-directory: ./build
- name: Test
run: ctest -j2 --output-on-failure -E linkchecker
run: ctest -j$(nproc) --output-on-failure -E linkchecker
working-directory: ./build
- name: Build Package Source
run: ninja -j2 package_source
run: ninja -j$(nproc) package_source
working-directory: ./build
- name: 'Upload Artifact'
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v4
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/release.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,16 @@ jobs:
run: cmake ..
working-directory: ./build
- name: Build All
run: make -j2 all
run: make -j$(nproc) all
working-directory: ./build
- name: Build ZIP
run: make -j2 zipfile
run: make -j$(nproc) zipfile
working-directory: ./build
- name: Test
run: ctest -j2 --output-on-failure -E linkchecker
run: ctest -j$(nproc) --output-on-failure -E linkchecker
working-directory: ./build
- name: Build Package Source
run: make -j2 package_source
run: make -j$(nproc) package_source
working-directory: ./build
- name: Set Version
id: set_version
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/srg-mapping-table.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
run: cmake .. -G Ninja
working-directory: ./build
- name: Build
run: ninja -j2 rhel10 rhel9 ocp4
run: ninja -j$(nproc) rhel10 rhel9 ocp4
working-directory: ./build
- name: Build rule dir json
run: python3 utils/rule_dir_json.py
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/stabilize.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,20 @@ jobs:
run: cmake -DSSG_OVAL_SCHEMATRON_VALIDATION_ENABLED=OFF -DANSIBLE_CHECKS=ON -DENABLE_SCAPVAL13=ON -DSCAPVAL_PATH='/opt/scapval/SCAP-Content-Validation-Tool-1.3.5/scapval-1.3.5.jar' ..
working-directory: ./build
- name: Build All
run: make -j2 all
run: make -j$(nproc) all
working-directory: ./build
- name: Get SCAPVAL
run: wget $SCAPVAL_URL/$SCAPVAL_FILENAME.zip
- name: Unpack SCAPVAL
run: mkdir -p /opt/scapval/ && unar $SCAPVAL_FILENAME.zip -o /opt/scapval/
- name: Run SCAPVal
# Runs SCAPVal on all built datastream
run: ctest -j2 -R scapval --output-on-failure
run: ctest -j$(nproc) -R scapval --output-on-failure
- name: Lint Check
# Performs ansible-lint and yamllint checks on generated ansible playbooks
run: ctest -j2 -R ansible-playbook --output-on-failure
run: ctest -j$(nproc) -R ansible-playbook --output-on-failure
working-directory: ./build
- name: Link Check
# Performs linkcheck across all build tables and html guides to ensure there are no broken references.
run: ctest -j2 -R linkchecker --output-on-failure
run: ctest -j$(nproc) -R linkchecker --output-on-failure
working-directory: ./build
Loading