[#697] Github actions to run all PRC tests #772

d-w-moore · 2025-09-29T14:17:29Z

All tests will run from GitHub actions.

Except the interactive PAM test, and that omission will soon be rectified.

.github/workflows/run-bats-tests.yml

irods/test/harness/build-docker.sh

test_harness/single_node/docker_container_driver.sh

test_harness/single_node/install.sh

d-w-moore · 2025-10-09T17:01:34Z

All the checks implemented via github actions (except Codacy) are passing . Some cleanup has been done on the code as well. I think we're for a preliminary round of review. Just be aware there are some extra scripts that are not central to the github actions we've implemented here. They are there as examples and tests of the harness functionality.

trel · 2025-10-09T18:36:52Z

please mark the ones that are 'not central'.

are you suggesting we keep the scripts that are 'not central'? if so, why?

d-w-moore · 2025-10-09T18:59:28Z

please mark the ones that are 'not central'.

are you suggesting we keep the scripts that are 'not central'? if so, why?

They're illustrative but not necessary, so I'll remove them.

trel · 2025-10-09T19:00:48Z

They're illustrative but not necessary, so I'll remove them.

Ah, okay - yes please remove them. And consider putting any helpful words / examples in README.md files at the appropriate locations.

d-w-moore · 2025-10-10T16:06:15Z

We are now down to 6 basic commits.

d-w-moore · 2025-10-13T15:31:02Z

We're now down to 8 basic commits. If that's too many for the convenience of those reviewing, let me know and I can maybe digest it down further.

korydraughn · 2025-10-13T15:47:36Z

8 commits isn't bad at all, but let's go ahead and squash commits into their final shape if we know what that is.

d-w-moore · 2025-10-13T17:11:43Z

Commits now minimized to 2.

.github/workflows/README.md

docker-testing/recv_oneshot

docker-testing/run_tests.sh

docker-testing/send_oneshot

.github/workflows/README.md

.github/workflows/run-the-tests.yml

.github/workflows/README.md

docker-testing/README.md

irods/test/harness/install.sh

test_harness/single_node/install_python_rule_engine

irods/test/scripts/test_support_functions

irods/test/harness/000_install-irods.Dockerfile

irods/test/harness/001_bats-python3.Dockerfile

irods/test/harness/003_compile-specific-python.Dockerfile

test_harness/single_node/README.md

irods/test/harness/README.txt

irods/test/scripts/test001_pam_password_expiration.bats

irods/test/scripts/test010_issue_362_rogue_chars_in_pam_password.bats

test_harness/multiple_node/README.md

docker-testing/irods_catalog_provider/entrypoint.sh

docker-testing/run_tests.sh

irods/test/harness/001_bats-python3.Dockerfile

.github/workflows/README.md

irods/test/scripts/test003_write_pam_credentials_to_secrets_file.bats

irods/test/scripts/test010_issue_362_rogue_chars_in_pam_password.bats

irods/test/scripts/test_support_functions

irods/test/scripts/iinit.py

irods/test/login_auth_test.sh

d-w-moore · 2025-11-14T17:28:43Z

Reverted all experimental changes and things are back to normal for final round of commentary and review....

trel · 2025-11-14T17:42:27Z

please squash so this is a candidate for merging. i still see two extra commits i think.

and consider renaming irods/test/setupssl.py to irods/test/setup_ssl.py

d-w-moore · 2025-11-14T18:02:55Z

Should f59678c and ff808aa be squashed together?

My own editorial on it, keep them separate for proper attribution. And, because there is fundamentally different work being done:

the first commit is basically the development of the github infrastructure and the multiple-node harness.
the second commit is the single-node harness and related changes to the setup scripts for the tests themselves.

alanking · 2025-11-14T21:55:31Z

My top suggestion at this point is to revert / delete that last commit. Why rename setupssl.py? I think we should leave that alone since it's publicly available and documented as setupssl.py. If you want to rename it, let's make a new issue and discuss how to do that at another time... unless you need to rename it in order for these changes to work.

I think the "remove Jenkins test framework support" commit can be linked to #778, yes?

Finally, looking at "[_TENTATIVE_SQUASH] mostly review based changes ..."...

This one should be split up and squashed into the commit(s) with which each change is best suited. For example, it includes a change to get rid of all symlinks. I see one symlink that was introduced in "test harness and container-based tests", so that should be squashed into there.

If no reasonable (I will let you decide what that means) split / squash can be had for these changes, maybe just keep it in its own commit and link it to all of the relevant issues. It's not ideal, but if it can't be done in a clean manner, maybe that's all we can do.

I think the rest of the commits look fine.

trel · 2025-11-14T21:57:00Z

the rename ssl thing was me - i noticed all the other files had underscores in their naming... and it stood out.

d-w-moore · 2025-11-14T23:31:39Z

Finally, looking at "[_TENTATIVE_SQUASH] mostly review based changes ..."...

This one should be split up and squashed into the commit(s) with which each change is best suited. For example, it includes a change to get rid of all symlinks. I see one symlink that was introduced in "test harness and container-based tests", so that should be squashed into there.

If no reasonable (I will let you decide what that means) split / squash can be had for these changes, maybe just keep it in its own commit and link it to all of the relevant issues.

I can split it up pretty easily, as it still exists on another branch in unsplit form.

d-w-moore · 2025-11-14T23:32:43Z

the rename ssl thing was me - i noticed all the other files had underscores in their naming... and it stood out.

I'll work on the merge-compatible squashing tonight and defer to consensus on the renaming or not of setupssl.py

korydraughn

Almost there!

irods/test/scripts/test_support_functions

irods/test/login_auth_test.sh

test_harness/single_node/setup_python_rule_engine

test_harness/single_node/README.md

alanking · 2025-11-17T13:45:45Z

Given the new changes and the way things are arranged now, I would make a best-effort attempt at squashing the commits without issue numbers into the commits with issue numbers. If it proves too messy/difficult to sort through, I (personally) am not opposed to "formatting" commits (which exclusively or primarily deal with whitespace) that come at the end of a line of commits. Leaving that up to you, @d-w-moore.

the rename ssl thing was me - i noticed all the other files had underscores in their naming... and it stood out.

Looking into this further, I think it's okay to rename that script. It appears to be used internally for testing only. Please squash that into the commits with issue numbers as well.

d-w-moore · 2025-11-17T16:06:53Z

... I would make a best-effort attempt at squashing the commits without issue numbers into the commits with issue numbers. If it proves too messy/difficult to sort through, I (personally) am not opposed to "formatting" commits (which exclusively or primarily deal with whitespace) that come at the end of a line of commits. Leaving that up to you, @d-w-moore.

the rename ssl thing was me - i noticed all the other files had underscores in their naming... and it stood out.

Looking into this further, I think it's okay to rename that script. It appears to be used internally for testing only. Please squash that into the commits with issue numbers as well.

No problems, will do the squashing next.

korydraughn

Pound it!

Co-authored-by: d-w-moore <dmoore@renci.org> Includes a docker compose configuration in which the PRC test suite can be run in a way representative of typical operational use, and the version of iRODS server and python interpreter that we install are also easily reconfigurable. Additionally, this affords us the opportunity to run the test suite on a client node all its own, the iRODS server being reachable only via the network.

A new test harness is introduced in which we construct a new container (using either Docker or podman) for each test program we run. This allows full customization of the container environment for the particular needs of each test. Accordingly, also included with the Github workflows is a full run of the PRC test suite with the iRODS server and catalog DB server running in the same container as the client. In the process of putting old tests through new rigors, faults were found and corrected in some of those tests.

The test now fetches the modify and access timestamps during the interval in which the replica is open, then asserts those timestamps as equal to each other, as well as that they are greater than a system-generated datetime of about two seconds prior. (This is due to some timestamps including a microseconds count whereas others do not.)

alanking · 2025-11-24T17:08:14Z

The documentation introduced in this PR for running the tests also addresses #637. We will not be linking any specific commit to the issue, so I am doing it here in this comment.

Merging.

d-w-moore marked this pull request as draft September 29, 2025 14:17

d-w-moore force-pushed the ghrunner-run-all-tests branch 2 times, most recently from 9bd1449 to 19ef753 Compare October 3, 2025 20:33

d-w-moore commented Oct 4, 2025

View reviewed changes

.github/workflows/run-bats-tests.yml Outdated Show resolved Hide resolved

d-w-moore commented Oct 4, 2025

View reviewed changes

irods/test/harness/build-docker.sh Outdated Show resolved Hide resolved

d-w-moore commented Oct 4, 2025

View reviewed changes

test_harness/single_node/docker_container_driver.sh Show resolved Hide resolved

d-w-moore commented Oct 4, 2025

View reviewed changes

test_harness/single_node/install.sh Show resolved Hide resolved

d-w-moore force-pushed the ghrunner-run-all-tests branch 4 times, most recently from fa0b1ae to 5cfcce8 Compare October 5, 2025 04:34

d-w-moore marked this pull request as ready for review October 9, 2025 17:01

d-w-moore self-assigned this Oct 9, 2025

d-w-moore force-pushed the ghrunner-run-all-tests branch from 08a524c to 26cc9a1 Compare October 10, 2025 16:05

d-w-moore force-pushed the ghrunner-run-all-tests branch 2 times, most recently from f14118d to e2f2615 Compare October 13, 2025 15:28

d-w-moore force-pushed the ghrunner-run-all-tests branch from e2f2615 to a395ff6 Compare October 13, 2025 17:10

korydraughn reviewed Oct 13, 2025

View reviewed changes

trel reviewed Oct 13, 2025

View reviewed changes

korydraughn reviewed Oct 13, 2025

View reviewed changes

alanking reviewed Oct 14, 2025

View reviewed changes

d-w-moore closed this Oct 17, 2025

korydraughn reviewed Nov 14, 2025

View reviewed changes

d-w-moore force-pushed the ghrunner-run-all-tests branch 4 times, most recently from 684c2c5 to 2108494 Compare November 17, 2025 12:39

d-w-moore force-pushed the ghrunner-run-all-tests branch 3 times, most recently from af5d51b to b027915 Compare November 18, 2025 02:34

alanking approved these changes Nov 21, 2025

View reviewed changes

korydraughn approved these changes Nov 21, 2025

View reviewed changes

trel and others added 6 commits November 21, 2025 21:49

[irods#778] remove Jenkins test framework support

45dfe92

[#3] rename setupssl.py to setup_ssl.py for consistency

89375e9

[irods#783] add pyproject.toml

14a3895

d-w-moore force-pushed the ghrunner-run-all-tests branch from 24d8f7e to 14a3895 Compare November 22, 2025 02:53

d-w-moore mentioned this pull request Nov 24, 2025

Document how to run the test suite #637

Closed

alanking merged commit 94b8594 into irods:main Nov 24, 2025
13 of 14 checks passed

alanking mentioned this pull request Nov 24, 2025

add at least a nominal pyproject.toml in root of repository #783

Closed

[#697] Github actions to run all PRC tests #772

[#697] Github actions to run all PRC tests #772

Uh oh!

Conversation

d-w-moore commented Sep 29, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

d-w-moore commented Oct 9, 2025

Uh oh!

trel commented Oct 9, 2025

Uh oh!

d-w-moore commented Oct 9, 2025

Uh oh!

trel commented Oct 9, 2025

Uh oh!

d-w-moore commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

d-w-moore commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

korydraughn commented Oct 13, 2025

Uh oh!

d-w-moore commented Oct 13, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

d-w-moore commented Nov 14, 2025

Uh oh!

trel commented Nov 14, 2025

Uh oh!

d-w-moore commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alanking commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trel commented Nov 14, 2025

Uh oh!

d-w-moore commented Oct 10, 2025 •

edited

Loading

d-w-moore commented Oct 13, 2025 •

edited

Loading

d-w-moore commented Nov 14, 2025 •

edited

Loading

alanking commented Nov 14, 2025 •

edited

Loading

d-w-moore commented Nov 17, 2025 •

edited

Loading