-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add select on read for MWA correlator FITS files #1458
Conversation
cdfc0e3
to
ae54798
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1458 +/- ##
==========================================
- Coverage 99.93% 99.93% -0.01%
==========================================
Files 62 63 +1
Lines 21756 21748 -8
==========================================
- Hits 21741 21733 -8
Misses 15 15 ☔ View full report in Codecov by Sentry. |
f4add86
to
a244fce
Compare
a244fce
to
6ae87bc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do feel like a lot of the mwa specific stuff is lost on me, but I do have a question after a quick read
4a65b56
to
ca32d4c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the little change. Looks good to me
refactor some more code into utils methods to allow reuse
for more information, see https://pre-commit.ci
f71a6c6
to
f41752c
Compare
Description
Add support for select on read to MWA correlator FITS files to save memory and time. This adds support for most but not all select keywords. The implementation is a bit complicated because of the structure of MWA correlator FITS files, which is also different between the files written by the legacy correlator and MWAX.
Edit: we have benchmarking now and the benchmarks didn't slow down significantly
I am a bit concerned that some of these changes may have slowed down the reading when not selecting on read. I think this needs to be tested/profiled more, would love some help on that front.I also refactored some selection code for the baseline-time axis and the frequency axis into functions for better code reuse, leading to changes in
UVData.select
andUVCal.select
. And I improved the way the history string is built up.Motivation and Context
closes #1386
Types of changes
Checklist:
New feature checklist: