Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adds module path test #46

Merged
merged 4 commits into from
Sep 20, 2024
Merged

Adds module path test #46

merged 4 commits into from
Sep 20, 2024

Conversation

RuiApostolo
Copy link
Contributor

Also small fix in cirrus config, small change in black linter config, and small linting changes in a few .py files.

Should close #5

@RuiApostolo RuiApostolo added bug Something isn't working enhancement New feature or request labels Sep 6, 2024
@RuiApostolo RuiApostolo self-assigned this Sep 6, 2024
@RuiApostolo
Copy link
Contributor Author

Test run outputs:

Cirrus:

rapostol@cirrus-login2:~/work/reframe_dev$ reframe -r -C configuration/cirrus.py -c tests/modules/module_paths.py
[ReFrame Setup]
  version:           4.6.0-dev.1
  command:           '/work/y07/shared/cirrus-software/reframe/bin/reframe -r --keep-stage-files -C configuration/cirrus.py -c tests/modules/module_paths.py'
  launched by:       rapostol@cirrus-login2
  working directory: '/mnt/lustre/e1000/home/z04/z04/rapostol/reframe_dev'
  settings files:    '<builtin>', 'configuration/cirrus.py'
  check search path: '/mnt/lustre/e1000/home/z04/z04/rapostol/reframe_dev/tests/modules/module_paths.py'
  stage directory:   '/mnt/lustre/e1000/home/z04/z04/rapostol/reframe_dev/stage'
  output directory:  '/mnt/lustre/e1000/home/z04/z04/rapostol/reframe_dev/output'
  log files:         '/mnt/lustre/e1000/home/z04/z04/rapostol/reframe_dev/reframe.out', '/mnt/lustre/e1000/home/z04/z04/rapostol/reframe_dev/reframe.log'

[==========] Running 2 check(s)
[==========] Started on Fri Sep  6 16:48:15 2024+0100

[----------] start processing checks
[ RUN      ] ModulePathTestCirrus %path=/usr/share/Modules/modulefiles /17f54bb5 @cirrus:login+Default
[ RUN      ] ModulePathTestCirrus %path=/work/y07/shared/cirrus-modulefiles /eb133a02 @cirrus:login+Default
[       OK ] (1/2) ModulePathTestCirrus %path=/usr/share/Modules/modulefiles /17f54bb5 @cirrus:login+Default
[       OK ] (2/2) ModulePathTestCirrus %path=/work/y07/shared/cirrus-modulefiles /eb133a02 @cirrus:login+Default
[----------] all spawned checks have finished

[  PASSED  ] Ran 2/2 test case(s) from 2 check(s) (0 failure(s), 0 skipped, 0 aborted)
[==========] Finished on Fri Sep  6 16:48:16 2024+0100
Log file(s) saved in '/mnt/lustre/e1000/home/z04/z04/rapostol/reframe_dev/reframe.out', '/mnt/lustre/e1000/home/z04/z04/rapostol/reframe_dev/reframe.log'

ARCHER2:

rapostol@ln02:~/work/epcc-reframe> reframe -r -C configuration/archer2.py -c tests/modules/module_paths.py
[ReFrame Setup]
  version:           4.2.1
  command:           '/work/y07/shared/utils/core/reframe/4.2.1/bin/reframe --keep-stage-files -r -C configuration/archer2.py -c tests/modules/module_paths.py'
  launched by:       rapostol@ln02
  working directory: '/mnt/lustre/a2fs-work2/work/z19/z19/rapostol/epcc-reframe'
  settings files:    '<builtin>', 'configuration/archer2.py'
  check search path: '/mnt/lustre/a2fs-work2/work/z19/z19/rapostol/epcc-reframe/tests/modules/module_paths.py'
  stage directory:   '/mnt/lustre/a2fs-work2/work/z19/z19/rapostol/epcc-reframe/stage'
  output directory:  '/mnt/lustre/a2fs-work2/work/z19/z19/rapostol/epcc-reframe/output'
  log files:         '/mnt/lustre/a2fs-work2/work/z19/z19/rapostol/epcc-reframe/reframe.out', '/mnt/lustre/a2fs-work2/work/z19/z19/rapostol/epcc-reframe/reframe.log'

[==========] Running 17 check(s)
[==========] Started on Fri Sep  6 16:44:53 2024

[----------] start processing checks
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/modulefiles /cb0ac43d @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/usr/local/share/modulefiles /5af26961 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/craype-targets/default /38554b7f @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/core /239d3636 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/lmod/modulefiles/Core /c13957c7 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/cpu/x86-rome/1.0 /b3ff4ae1 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/net/ofi/1.0 /1e34c62a @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/perftools/22.12.0 /a4a897b5 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/mix_compilers /80c09dd8 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/compiler/crayclang/14.0 /60552a03 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/comnet/crayclang/14.0/ofi/1.0 /0ac1e88d @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/mpi/crayclang/14.0/ofi/1.0/cray-mpich/8.0 /9c8d293a @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/utils/core /1a9f2e26 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/apps/core /c224e69b @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/libs/core /4fe98ce6 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/python/core /b9bb3b53 @archer2:login+Default
[ RUN      ] ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/utils/compiler/crayclang/10.0 /29ce6a97 @archer2:login+Default
[       OK ] ( 1/17) ModulePathTestARCHER2 %path=/opt/cray/modulefiles /cb0ac43d @archer2:login+Default
[       OK ] ( 2/17) ModulePathTestARCHER2 %path=/usr/local/share/modulefiles /5af26961 @archer2:login+Default
[       OK ] ( 3/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/craype-targets/default /38554b7f @archer2:login+Default
[       OK ] ( 4/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/core /239d3636 @archer2:login+Default
[       OK ] ( 5/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/lmod/modulefiles/Core /c13957c7 @archer2:login+Default
[       OK ] ( 6/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/cpu/x86-rome/1.0 /b3ff4ae1 @archer2:login+Default
[       OK ] ( 7/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/net/ofi/1.0 /1e34c62a @archer2:login+Default
[       OK ] ( 8/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/perftools/22.12.0 /a4a897b5 @archer2:login+Default
[       OK ] ( 9/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/mix_compilers /80c09dd8 @archer2:login+Default
[       OK ] (10/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/compiler/crayclang/14.0 /60552a03 @archer2:login+Default
[       OK ] (11/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/comnet/crayclang/14.0/ofi/1.0 /0ac1e88d @archer2:login+Default
[       OK ] (12/17) ModulePathTestARCHER2 %path=/opt/cray/pe/lmod/modulefiles/mpi/crayclang/14.0/ofi/1.0/cray-mpich/8.0 /9c8d293a @archer2:login+Default
[       OK ] (13/17) ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/utils/core /1a9f2e26 @archer2:login+Default
[       OK ] (14/17) ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/apps/core /c224e69b @archer2:login+Default
[       OK ] (15/17) ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/libs/core /4fe98ce6 @archer2:login+Default
[       OK ] (16/17) ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/python/core /b9bb3b53 @archer2:login+Default
[       OK ] (17/17) ModulePathTestARCHER2 %path=/work/y07/shared/archer2-lmod/utils/compiler/crayclang/10.0 /29ce6a97 @archer2:login+Default
[----------] all spawned checks have finished

[  PASSED  ] Ran 17/17 test case(s) from 17 check(s) (0 failure(s), 0 skipped, 0 aborted)
[==========] Finished on Fri Sep  6 16:44:56 2024
Log file(s) saved in '/mnt/lustre/a2fs-work2/work/z19/z19/rapostol/epcc-reframe/reframe.out', '/mnt/lustre/a2fs-work2/work/z19/z19/rapostol/epcc-reframe/reframe.log'

@JPRichings
Copy link
Contributor

There are a couple of items in module path on both systems we need to tidy up as they don't get used but aren't tested here.

Archer2 is missing paths currently in MODULEPATH but not actually containing modules:

echo $MODULEPATH | tr ":" "\n"
/work/y07/shared/archer2-lmod/utils/compiler/crayclang/10.0
/work/y07/shared/archer2-lmod/training/core
/work/y07/shared/archer2-lmod/python/pyenvs
/work/y07/shared/archer2-lmod/python/core
/work/y07/shared/archer2-lmod/libs/core
/work/y07/shared/archer2-lmod/apps/core
/work/y07/shared/archer2-lmod/utils/core
/opt/cray/pe/lmod/modulefiles/mpi/crayclang/14.0/ofi/1.0/cray-mpich/8.0
/opt/cray/pe/lmod/modulefiles/comnet/crayclang/14.0/ofi/1.0
/opt/cray/pe/lmod/modulefiles/compiler/crayclang/14.0
/opt/cray/pe/lmod/modulefiles/mix_compilers
/opt/cray/pe/lmod/modulefiles/perftools/22.12.0
/opt/cray/pe/lmod/modulefiles/net/ofi/1.0
/opt/cray/pe/lmod/modulefiles/cpu/x86-rome/1.0
/opt/cray/pe/modulefiles/Linux
/opt/cray/pe/modulefiles/Core
/opt/cray/pe/lmod/lmod/modulefiles/Core
/opt/cray/pe/lmod/modulefiles/core
/opt/cray/pe/lmod/modulefiles/craype-targets/default
/usr/local/share/modulefiles
/opt/cray/modulefiles

The four not currently tests or used but in MODULEPATH are:

/work/y07/shared/archer2-lmod/training/core (directory doesn't exist but the one above does)
/work/y07/shared/archer2-lmod/python/pyenvs (directory doesn't exist)
/opt/cray/pe/modulefiles/Linux (directory doesn't exist)
/opt/cray/pe/modulefiles/Core (directory doesn't exist)

Cirrus is similar and is missing two paths:

echo $MODULEPATH | tr ":" "\n"
/work/y07/shared/cirrus-modulefiles
/usr/share/Modules/modulefiles
/etc/modulefiles
/usr/share/modulefiles
[jpr@cirrus-login3 epcc-reframe]$

The two unused but missing from the test:

/etc/modulefiles
/usr/share/modulefiles

@JPRichings
Copy link
Contributor

I tested this and is does confirm all the module directories exist correctly.

We could expand this to test that the module command has correctly picked up these directories and not just that they exist.

@JPRichings JPRichings merged commit 518b122 into EPCCed:main Sep 20, 2024
9 checks passed
@RuiApostolo RuiApostolo deleted the 20240906_module_paths branch September 20, 2024 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JPRichings JPRichings Awaiting requested review from JPRichings

Assignees

@RuiApostolo RuiApostolo

Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add module tests
2 participants
@RuiApostolo @JPRichings