Non-blocking PUT in CHPL_COMM=ofi #25977
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Previously, non-blocking PUTs were implemented via blocking PUTs, which could severely limit performance. Prior to 2.0, small PUTs invoked fi_inject_write, which essentially turned them into non-blocking PUTs, but chpl_comm_put returned as if the PUT was completed. This could cause MCM violations as well as hangs caused by not progressing the network stack properly. These deficiences were fixed in 2.0, but led to a performance regression. This commit implements non-blocking PUTs correctly, so that the chpl_comm_*nb* functions work correctly. This should restore 1.32.0 performance while avoiding MCM violations and hangs. Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Rewrote PUT logic so that low-level functions are non-blocking, and a blocking PUT is implemented by initiating a non-blocking PUT and waiting for it to complete. This simplifies the implementation and avoids code duplication. Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Allow specifying the maximum message size and maximum number of endpoings. These are intended primarily for testing. Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Also some code cleanup. Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
We are now using this function to force visibility when an unbound endpoint is released, so it needs to work on unbound endpoints. Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Operations to force visibility are deferred until the endpoint is released, which requires the visibility bitmaps. Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Fixed how the number of transmit contexts needed is computed, and added some comments. Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Change type of numTxCtxs and numRxCtxs to size_t to match type of info->domain_attr->ep_cnt. Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Signed-off-by: John H. Hartman <jhh67@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TODO: flesh this out
Proper implementation of non-blocking PUT in
CHPL_COMM=ofi.