Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-design CommunicatorGrid/Communicator/Pipeline interaction #430

Closed
albestro opened this issue Nov 30, 2021 · 2 comments
Closed

Re-design CommunicatorGrid/Communicator/Pipeline interaction #430

albestro opened this issue Nov 30, 2021 · 2 comments

Comments

@albestro
Copy link
Collaborator

With #423 we removed the synchronization point at the end of each algorithm due to the Pipeline<Communicator>, but we had to introduce comm.clone() for each Pipeline at the beginning of each algorithm in order to keep Communicators independent.

One of the first seed for ideas from the team is to allocate a pool of Communicators associated 1:1 with Pipelines, providing parallelism and safety. Indeed, in this way we will have a restricted set of Communicators, which allows to keep their number limited avoiding an "overload" of MPI while still allowing independent communications. On the other hand, their associated Pipelines ensure that inside the associated Communicator communications are ordered even among different algorithms, because the Pipeline "keeps the history" of the operations and acts as "global" manager for all the algorithms sharing the same Pipeline/Communicator pair.

The starting point seems to be embedding the Pipeline in the Communicator or in the CommunicatorGrid, so that each algorithm can request access to the communicator and it will get its associated Pipeline.

@rasolca
Copy link
Collaborator

rasolca commented May 16, 2023

See https://confluence.cscs.ch/x/hYBZHw for details.

@msimberg msimberg self-assigned this Sep 18, 2023
@msimberg
Copy link
Collaborator

Closed by #993.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

3 participants