Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clarify and implement the precedences of I/O hints #173

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

wkliao
Copy link
Member

@wkliao wkliao commented Feb 28, 2025

Clarify and implement a new I/O hint precedences

* 1st priority: hints set in the environment variable `PNETCDF_HINTS`, e.g.
  `PNETCDF_HINTS="nc_var_align_size=1024"`. Making this the first priority
  is because it allows the same application executable without source code
  modification to run using different settings through a run-time
  environment varaible.
* 2nd priority: hints used in the arguments of `ncmpi__enddef()`, e.g.
  `ncmpi__enddef(..., v_align=1024,...)`. With the same reason as described
  above, application source codes making a call to APIs using specific
  arguments should have a priority lower than the ones set in a run-time
  environment variable, so that users can try different hints using the
  same executable.
* 3rd priority: hints set in the MPI info object passed into calls of
  `ncmpi_create()` and `ncmpi_open()`, e.g.
  `MPI_Info_set("nc_var_align_size", "1024");`. This is because MPI info
  can only be passed at the file creation or open time whose hints are
  generally applied to the whole operation to the fille until it is closed.
  However, `ncmpi__endde()` can be called multiple times between a file's
  open and close, as users may want to use a settings unique for individual
  call to `ncmpi__endde()`. Thus `ncmpi__endde()` should have a higher
  priority than MPI info.

@wkliao wkliao changed the title Rm nc header align size clarify of nc_header_align_size Feb 28, 2025
@wkliao wkliao changed the title clarify of nc_header_align_size clarify and implement the precedences of I/O hints Feb 28, 2025
wkliao added 4 commits March 5, 2025 15:51
* 1st priority: hints set in the environment variable `PNETCDF_HINTS`,
  e.g. `PNETCDF_HINTS="nc_var_align_size=1024"`
* 2nd priority: hints passed from arguments of `ncmpi__enddef()`, e.g.
  `ncmpi__enddef(..., v_align=1024,...)`
* 3rd priority: hints set in the MPI info objects passed into calls to
  `ncmpi_create()` and `ncmpi_open()`, e.g.
  `MPI_Info_set("nc_var_align_size", "1024");`

When both hints nc_var_align_size and nc_header_align_size are set in
PNETCDF_HINTS or MPI info object, only nc_var_align_size will take
effect.

When there is no fix-sized variables defined, hint v_minfree is ignored.
In this case, users should set h_minfree, if extra space is desired.

When there is no fix-sized variables defined and v_align is not set,
hint r_align, if set, will be used as v_align.
Because nc_header_align_size serves exactly the same purpose of
nc_var_align_size and when both are set by the users, only
nc_var_align_size takes effect.
@wkliao wkliao force-pushed the rm_nc_header_align_size branch from ab2120d to cee0663 Compare March 5, 2025 22:18
@wkliao wkliao force-pushed the rm_nc_header_align_size branch from c111634 to 4d888f4 Compare March 9, 2025 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant