Reduction binary ops on floating-points (and NaN) might not be strictly associative/commutative

[davidozog]

Problem Statement

From the OpenSHMEM v1.6 project, we encountered this issue:
https://github.com/orgs/openshmem-org/projects/5/views/1?pane=issue&itemId=74045800

Reduction binary ops on floating-points (and NaN) might not be strictly associative/commutative.

Proposed Changes/Ideas

• The result should be the same on all PEs.
• The result should be the same from run to run.
• The result may be different on different architectures, due to differing arithmetic.
• The result on IEEE-754 compatible platforms should always be the same everywhere every time.
• We might consider new APIs or environment variables that enforce more consistent results.
• A team could take an optional config parameter indicating desire for a well-ordered reduction (i.e, C++ std::reduce vs std::accumulate)

References and Pull Requests

(original v1.6 project issue) https://github.com/orgs/openshmem-org/projects/5/views/1?pane=issue&itemId=74045800
(closed) davidozog#14
davidozog#16