Skip to content

Draft: parallelize match computation in pycbc_brute_bank by shrinking multiple templates #4815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

yi-fan-wang
Copy link
Member

Opening it here for discussion. I'd like to further parallelize pycbc_brute_bank wherever possible. The idea is to shrink multiple waveforms at once, as many as the parallel processes.

However, it doesn't really work as fast as I expected. It's actually much slower than a serial computation altogether. I suspect it's because of resource contention inside of the multiple processes.

I'd also like to explore the consequences to return inside one of the multiprocessing pools.

@ahnitz
Copy link
Member

ahnitz commented Jul 17, 2024

@yi-fan-wang I'm not sure this approach will work. I think the most straightforward is simple to parallelize over the proposals themselves and assume that within each proposal set there isn't much overlap.

@GarethCabournDavies
Copy link
Contributor

@yi-fan-wang should this be closed?

@yi-fan-wang yi-fan-wang closed this Jul 4, 2025
@yi-fan-wang
Copy link
Member Author

Leave some notes here for future development: For some unknown reason the parallelization doesn't work as I want. It may be that opening and closing the parallelization too frequently cost too much time. So maybe try to incorporating all things in a multiprocessing pool

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants