Add parallelism to searching for builds #970
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rydrman
reviewed
May 17, 2024
There was a problem hiding this comment.
I'm okay with these overall, actually - it seems like the update with arcswap is a little nicer than the leaky box but I can imagin how this might not have a huge impact
I was reading a little on io_uring the other day and I think it will be interesting to test once it settles into kernel versions that are not bleeding edge. There's some opinions about whether using async for i/o bound workloads can really improve any throughput. Not that I regret our use of async, but it could be true that it's not currently a viable path to much better performance
|
Okay, I'll freshen this up and be willing to merge it. |
These methods don't need `self`; remove it to make things easier for async code. Signed-off-by: J Robert Ray <jrray@jrray.org>
For better ergonomics compared to AtomicPtr, which required unsafe code. Signed-off-by: J Robert Ray <jrray@jrray.org>
The code does not require it; this change allows for the removal of DerefMut for SpfsRepository, which is needed for the next change. Signed-off-by: J Robert Ray <jrray@jrray.org>
Put the non-clone members behind an Arc. This change facilitates code that would want to move an SpfsRepository instance into an async task. Signed-off-by: J Robert Ray <jrray@jrray.org>
When checking for the existence of tags called 1, 1.0, 1.0.0, etc., spawn tasks for each form. Signed-off-by: J Robert Ray <jrray@jrray.org>
jrray
force-pushed
the
jrray/issue966-parallel
branch
from
7154581
to
71493ac
Compare
JoinSet is a better choice in a situation where tasks are getting spawned. Signed-off-by: J Robert Ray <jrray@jrray.org>
jrray
force-pushed
the
jrray/issue966-parallel
branch
from
64a5c26
to
6774c56
Compare
|
@rydrman I got this cleaned up, it can wait until after the holiday. |
rydrman
approved these changes
May 21, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'll say up front that I have doubts this change is actually going to improve performance since the overhead of #966 is mainly file IO.
I had originally tried to implement #966 in a more parallel way but ran into lifetime issues with the async tasks, as would be expected. This PR explores what was required to overcome those lifetime issues.
Some of the changes here might be worth cherry-picking and keeping.