Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply coadd to only non-zero pixels #787

Merged
merged 1 commit into from
Sep 24, 2024
Merged

Apply coadd to only non-zero pixels #787

merged 1 commit into from
Sep 24, 2024

Conversation

keskitalo
Copy link
Member

Only co-add nonzero pixels.

Copy link
Member

@tskisner tskisner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very specialized post-processing script, so whatever works in practice is great. Since every process has a copy of the hit portion of the map, as the number of hit pixels increases, the number of processes per node will be limited by the available memory (e.g imagine a footprint with half the sky at nside=16384).

If this ever becomes an issue, each of the current "tasks" could be converted to a process group with a PixelDistribution where the number of submaps is set to the number of processes in the group (where each process has one submap). Then all of the usual tools for loading data in parallel, inverting and applying covariance, etc can just be used without ever storing the whole map on a single process.

@keskitalo keskitalo merged commit 04fa343 into toast3 Sep 24, 2024
6 checks passed
@tskisner tskisner deleted the faster_coadd branch September 25, 2024 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants