Skip to content

Remove the dependency on CPAN module List::MoreUtils #337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 27, 2025

Conversation

underwoo
Copy link
Member

@underwoo underwoo commented Jan 3, 2025

Description
The Perl script split_ncvars.pl uses a CPAN module List::MoreUtils for a single function uniq. The use of this function can be removed using a better method to grab the intersection of two arrays:

my @array1 = ["val1", "val2", "val3"];
my @array2 = ["val1", "val3", "val4"];
my %arr1 = map { $_ => () } @array1
my @intersection = grep { exists $arr1{$_} } @array2;

This assumes the values in @array1 and @array2 are unique. Since the arrays in split_ncvars.pl hold dimensions, the lists should be unique.

How Has This Been Tested?
The split_ncvars.pl tests are run, which indicates the output is identical to previous runs.

This PR should be taken after #331.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules
  • New check tests, if applicable, are included
  • make distcheck passes

ceblanton
ceblanton previously approved these changes Jan 22, 2025
Copy link
Contributor

@ceblanton ceblanton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change to remove uniq looks good. The rest of the changes look like they were in #331.

@underwoo underwoo dismissed ceblanton’s stale review January 22, 2025 17:00

The merge-base changed after approval.

@ceblanton
Copy link
Contributor

@underwoo this looks good, but it's still in Draft. Is there anything else to add?

@underwoo underwoo marked this pull request as ready for review January 27, 2025 16:43
@ceblanton ceblanton self-requested a review January 27, 2025 21:29
ceblanton
ceblanton previously approved these changes Jan 27, 2025
@underwoo underwoo dismissed ceblanton’s stale review January 27, 2025 21:30

The merge-base changed after approval.

@mlee03 mlee03 requested a review from ceblanton January 27, 2025 21:31
ceblanton
ceblanton previously approved these changes Jan 27, 2025
@underwoo underwoo dismissed ceblanton’s stale review January 27, 2025 21:35

The merge-base changed after approval.

@mlee03 mlee03 self-requested a review January 27, 2025 21:36
mlee03
mlee03 previously approved these changes Jan 27, 2025
@mlee03 mlee03 dismissed their stale review January 27, 2025 21:41

The merge-base changed after approval.

@ceblanton ceblanton self-requested a review January 27, 2025 21:43
@mlee03 mlee03 merged commit 5d306c0 into NOAA-GFDL:main Jan 27, 2025
4 checks passed
mlee03 added a commit to mlee03/FRE-NCtools that referenced this pull request Jan 29, 2025
* Update land_utils testing and move to new layout

Updated the land_utils application testing to not require bats.  Included with this is a new testing framework, which was acquired from GNU coreutils.  This testing framework, while not as extensive as bats, has several built-in features that improve the testing process.

This update also begins the migration of the utilities out of the postpocessing and tools directories.

* Move mppnccombine to src directory

Add standard version text to -V option for mppnccombine

* Add Intel mac c configuration

Add tasks, and configure cspell settings

* Add some mppnccombine tests

* Renamed mppnccombine tests

* Some tests for mppnccombine

* working on iceberg_comb

Include print_version helper program

* New iceberg_comb.sh

* Renamed iceberg_comb.sh to have .in end for expansion of variables

* Update documentation (man), and add check for file with no icebergs

* Moved iceberg_comb to src dir

This has no changes, so it will not build

* Moved iceberg_comb to src directory

* Add iceberg tests, clean out the old Makefile.am

* Moved combine_restarts

Other fixes found during testing

* Added more man pages for moved programs now in src

* Moved list_ncvars

Other fixes to allow in-source builds

* move list_ncvars to src

* Addressed sh lint warnings in combine_restarts

* Move plevel to src only

* plevel now moved

Next the "cleanup"

* Fixes for plevel build

* Add mkdir to .F90_mod.mod rule

* add plevel.sh man page

* plevel.sh first test

* Finish plevel mods and tests

* add plevel help and version options

* update plevel to print history and fre-nctools version informaiton

Cleaned up comments in tests, and added helper routines for fortran and tests

* Update changelog file

* Add additional documentation to nct_nc_utils.F90

* migrate time_average to src directory

time_average adds provanance information to the output netcdf file

* Verify that NC* commands exist, and have the vars be precious

* Replace AC_PYTHON_MODULE with AX_PYTHON_MODULE

Fix typo in ncatted program name

* Add time average manual pages

Correct authorship for some files in the man pages, change bug report address to GitHub issue URL, and add copyright year

* Add time average tests

* Some initial modifications for split_ncvars.pl

* split_ncvars.pl after perltidy

* Last cleanup before migrating to src directory

* Moved split_ncvars to src directory

* Remove -Werror from automake options

This is until ubuntu has a newer version of automake

* Remove varlist.csh and old Makefile.am from split_ncvars

* Minor updates for split_ncvars

Add ncatted as modifiable in split_ncvars

* Clean out old tests that are now part of the new non-bats tests

* Remote nccmp requirement for tests

* Check for getrusage

* Remove more old mppnccombine test files

* Add second test-suite.log file in tests to upload to build artifact

* Update mppnccombine tests to better deal with failed tests

* Remove Exit line from some mppnccombine tests

* Fix failing tests for mppnccombine

* converted list_ncvars to a sh script

* Remove csh test, as csh is not in the test image

* Add exclude option to nc_cmp

* Add split_ncvars tests

* Do not install or test split_ncvars if required perl module not available

* Use correct perl in split_ncvars

* change all _acc to _gpu

* libfrencutils_gpu did not get staged with previous commit

* Add mppnccombine test to verify output file kind matches input file kind

* Add split_ncvars.pl man page

* Migrated more tests for make_hgrid, make_vgrid, make_topog, make_solo_mosaic, and make_coupler_mosaic

Added man pages for these items as well

Added option to skip long tests

Allow tests to find mpirun-like application

* Add init.cfg to extra dist files

* Set coupled-model as expensive

* Add first fregrid test

Same as previous test20

* Revert coupled-model make_copuler_mosaic to previous version

* Remove old fregrid test (tests 20, 24, 26)

* Fix a few errors in tests/mppnccombine/input-output-kinds

* Add check_mask man page

* Add two parallel man pages

* Added fregrid.txt

Changed one of the create input file programs, removing occam from the name

add new create_mask_topog.c

* Fix input-output-kinds test for when mppnccombine is updated

* Enable the input-output-kinds mppnccombine test

* Add check_mask tests

* remove old Test08 and Test09

* slight correction to check_mask man text

* Add additional ignore files in .gitignore

* Add fregrid remap tests non-stretch grid

* Remove old Test32

* Move test31 to fregrid/remap-file-stretchgrid

* Remove last part of Test31

* remove old Test14

* fix t/Makefile.am trailing backslash

* fix missed _acc in cofnigre.ac

* one more

* POSIX script changes

* Add latlon regrid test

* Update site-config env.sh for nco requirement

* Change several license names from FMS to FRE-NCtools

* Update configure.ac to 2024.05

* Update with-end-number

* Add cubed-sphere fregrid test

The file generated to regrid is probably not correct, as the final output looks odd

My guess is that is more on the file, and not on fregrid.

* Update site-config env.sh for nco requirement

* Change several license names from FMS to FRE-NCtools

* Update configure.ac to 2024.05

* Turn on remap-stretchgrid as expensive

* Stop using non-standard asprint in test program

* Correct error in test file generation codes

* Some vscode settings

* remove Test05

* Migrated Test06 - fregrid extrap

Made changes to other tests to allow a single file to be used in multiple tests.

* Migrate test07

* remove Test09 script

The test was removed from the Makefile already

* Migrated test for remap_land

* Removed temporary python file

* Remove old Test10

* Remote Test10 from t/Makefile.am

* migrate Test11 to new test framework

* Remove Test11 from t directory

* Add make_regional_mosaic man page

* migrate Test12 to new testing framework

* Remove Test12 script and data

* Fixes to make_regional_mosaic and input-output-kinds tests

* Migrate Test13 to new testing framework

* Remove Test13 from t directory

* Add mppncscatter man page

* Add make_quick_mosaic man page

* Remove tests that are no longer required

* Correct random test failure with regional mosaic

* Migrate river_regrid test to new test framework

* Remove un-needed steps in the make_quick_mosaic test

* Remove river_regrid test and files

* Add a realy hydrology test

Remove unneeded test files, and all files in the old t directory

Updated the CONTRIBUTING.md to remove the mention of bats

* More updates to CONTRIBUTING.md

* Fix remap-file-mpi test

* Fix remap-file-mpi test

* Add more workflow tests

* Remove parallel build from expensive checks -- likely will overrun the runner host

* Add simple_hydrology/river_regird as extra_dist file

* Add license header to new test c files.

* fixes for gaussian topog type option (NOAA-GFDL#340)

* set uninitialized value in create_gaussian_topog

* fix wrong variable in loop

* fix loop variable again

---------

Co-authored-by: rem1776 <Ryan.Mulhall@noaa.gov>

* Remove the dependency on CPAN module List::MoreUtils (NOAA-GFDL#337)

* Remove List::MoreUtils as a dependency

* Minor updates to split_ncvars.pl

* Use the POSIX shell version of list_ncvars in split_ncvars

* remove unneeded section

---------

Co-authored-by: Seth Underwood <Seth.Underwood@noaa.gov>
Co-authored-by: Seth Underwood <underwoo@underwoo.io>
Co-authored-by: mlee03 <Mikyung.Lee@lscamd50-d.gfdl.noaa.gov>
Co-authored-by: Chris Blanton <chris.blanton@noaa.gov>
Co-authored-by: HansVahlenkamp <Hans.Vahlenkamp@noaa.gov>
Co-authored-by: Ryan Mulhall <35538242+rem1776@users.noreply.github.com>
Co-authored-by: rem1776 <Ryan.Mulhall@noaa.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants