FAQ
Fortran has a rich syntax for image selectors that are used in coindexed access. In the fully general case of a coarray with corank > 1, the image selector includes a list of cosubscripts, and can also include zero or more of a team selector (TEAM= or TEAM_NUMBER=), a notify variable (NOTIFY=), or even a stat argument (STAT=):
The PRIF RMA put routines include _notify variants to handle the notify variable (NOTIFY=), because it affects the communication semantics of the coindexed put. However all the handling of cosubscripts has been factored out of the PRIF RMA subroutines, which only accept an image number relative to the initial team. That image number should be obtained through separate calls to PRIF (usually prif_initial_team_index).
To understand why, first it's important to understand that a cosubscript is not an image number. In general cosubscripts are multi-dimensional and a coarray descriptor may have any arbitrary cobounds (where in particular, the colbounds need not be equal to 1).
Before any coindexed access can be performed at runtime, the list of cosubscripts must be translated into the corresponding image number in the initial team. The provided cosubscripts must first be interpreted relative to the cobounds in the coarray descriptor, to compute a scalar 1-based image offset. Next, that image offset must be interpreted relative to the correct team, which is either provided explicitly in the image selector (TEAM= or TEAM_NUMBER=) or otherwise the current team, in order to identify the image number in the initial team for use in communication. This process can entail non-trivial computational overheads. If the prif RMA routines accepted cosubscript and team arguments, then those overheads would be inextricably baked into every coindexed access.
PRIF encapsulates the conversion of cosubscripts into initial team image index as a separate PRIF call, which enables some compiler optimization of the translation overheads. For example, the compiler could identify repeated use of the same arguments to the coindexed expression and perform the translation once, potentially amortizing the translation overhead across many coindexed accesses. In some cases it may even be possible for a sufficiently powerful analysis to optimize the image translation overheads away entirely (notably for the important special case of corank==1 when the analysis can also statically determine the selected team is the initial team).
These are needed whenever the user invokes a language feature that requires remapping the cobounds of an existing coarray.
This can optionally happen in a CHANGE TEAM construct, if the user included a coarray-association-list in the CHANGE TEAM invocation. However not every CHANGE TEAM needs this.
It may also be needed when invoking a procedure with a coarray dummy argument that specifies different cobounds. See F23 15.5.2.9 "Coarray dummy variables":
alias_lcobounds and alias_ucobounds determine the cobounds for the returned alias_handle. The alias is not bound to any particular team, although of course later indexing into the alias using prif_image_index() will be relative to some team.
Basically the coarray should be allocated by the initial team using a call analogous to:
prif_allocate_coarray(lcobounds=[1], ucobounds=[num_images()], size_in_bytes=sizeof(prif_critical_type) ) The compiler is free to either create one prif_critical_type coarray per CRITICAL construct, or take the simpler approach of having exactly one prif_critical_type coarray for the whole program and conflating all the CRITICAL constructs together (which oversynchronizes, but is still a valid implementation). I'd suggest the latter approach for simplicity in your initial implementation, especially since CRITICAL is not a heavily used feature.
The short answer is that if your code generation has no need for context information, then you can simply pass a null pointer for cdata.
The cdataargument is entirely for your own use (as the compiler writer and client of PRIF), and the contents are entirely up to you. The prif_co_reduce call accepts cdata which is an arbitrary 64-bit value that is entirely opaque to the PRIF library. The PRIF library is the one that later invokes the operation_wrapper callback while computing the reduction, and the PRIF library will pass the same cdata value back to your generated code as the fourth argument to that operation_wrapper invocation when the corresponding reduction operation callback needs to be invoked. The value is associated with a particular image's current invocation of prif_co_reduce. So for example if you needed to pass a pointer to a compiler-generated stack frame closure or other metadata from the code around the user's call to CO_REDUCE into the code which marshals the invocation of the user-provided operation callback, you could pass that pointer in the cdata argument. The second example in the prif_co_reduce section of the PRIF 0.5 specification shows how that could be used to support a user-provided operation over a (non-interoperable) derived type with length type parameters.
As another example, if the user-provided operation is known to be an interoperable procedure (or the compilers knows it can safely create a usable procedure pointer to it with a 64-bit representation), then you could pass that pointer in the cdata argument and invoke it from within the generated operation_wrapper.
The short answer is "We made these changes to ensure PRIF can be portably implemented for any Fortran-compliant compiler". We encountered problems in practice when attempting to implement prif_co_{min,max} as specified in PRIF 0.4 for character arguments with LLVM flang.
The Fortran-level CO_MIN and CO_MAX are required to accept arguments of any integer, real, or character type (including any non-interoperable type kind variants). It's worth noting that the actual computational requirements are rather different for value comparison operations between numerical types (which always have a static width, usually 1..8 bytes) and alphabetical comparison between character strings (which can have arbitrary dynamic length).
In PRIF 0.4 we conflated all these argument types together in a single PRIF entry point with a type(*) dummy argument, and also included non-interoperable inputs. The main problem with this approach is that Fortran provides no guaranteed portable way for a callee (the PRIF library implementation) to disambiguate the exact underlying type and type parameters (i.e. character string length) once it's been conflated into a type(*) dummy argument. This problem becomes even worse if the input is a non-interoperable numerical or non-interoperable character type.
PRIF 0.5 solves this problem via two changes:
- Constrain argument type for argument
atoprif_co_{min,max,sum}to only accept interoperable types. Compilers are not required to provide non-interoperable numerical or character types, but if they do thenprif_co_reducecan be called to support any reductions invoked on those. Given the element storage size andCFI_type_tcode (from a CFI descriptor), this is enough for the PRIF implementation to uniquely disambiguate a matching interoperable numerical type. - Statically separate character typed inputs into
prif_co_max_characterandprif_co_min_characterentry points. This makes it easy for the PRIF implementation ofco_{min,max}to disambiguate when it's asked to perform alphabetic character comparisons on dynamic-length strings, and should ensure it can correctly infer the character length type parameter by inspecting the CFI descriptor.
PRIF 0.5 also overhauled the interface to prif_co_reduce. The problems with the old interface are discussed in section III-G of our LLVM-HPC paper. In a nutshell, the user-provided reduction callback function in general can accept non-interoperable types and might not be bind(C). For both reasons, the user-provided callback cannot be safely/portably invoked from C code in a type-erased manner based on a type(*) dummy argument, it must be invoked from Fortran code with properly concretely typed actual arguments. The PRIF interface deliberately omits detailed Fortran-level type information, and the PRIF implementation cannot generate Fortran caller code with argument types matching user-defined derived types, thus the compiler must take on that responsibility. The new operation_wrapper argument ensures the call to the user-provided function is made from Fortran with properly matching concrete actual argument types. We realize this complicates the calling convention for prif_co_reduce, but we felt it was important to ensure PRIF implementations can strictly adhere to the Fortran standard.
With these interface changes in place, we believe the C interop features in the Fortran standard guarantee sufficient information so that the PRIF collective subroutines can be portably implemented for any compliant compiler.