-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DM-43712: Add configurable buffer for template and refcat preload #204
base: main
Are you sure you want to change the base?
Commits on Sep 27, 2024
-
Update unit tests with the test data change
This should have been together with 674a4b1 Unlike the old DECam test dataset, the new test LSSTComCamSim dataset does not have crosstalk. So, use another dataset type to test. Also fix a time stamp missed in the previous commit.
Configuration menu - View commit details
-
Copy full SHA for 30770f0 - Browse repository at this point
Copy the full SHA 30770f0View commit details
Commits on Oct 2, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 6e01665 - Browse repository at this point
Copy the full SHA 6e01665View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0bb8cb6 - Browse repository at this point
Copy the full SHA 0bb8cb6View commit details -
Make the preload padding configurable at the service level
The optimal value likely depends on the instrument and the skymap choice.
Configuration menu - View commit details
-
Copy full SHA for 67bd7f1 - Browse repository at this point
Copy the full SHA 67bd7f1View commit details -
Use the instrument's formatter to get the sky wcs
Instead of determining the instrument's wcsFlipX here, it is more robust to use its formatter, which knows its camera orientation from its obs package.
Configuration menu - View commit details
-
Copy full SHA for 898fa56 - Browse repository at this point
Copy the full SHA 898fa56View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2696a76 - Browse repository at this point
Copy the full SHA 2696a76View commit details
Commits on Oct 4, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f97422c - Browse repository at this point
Copy the full SHA f97422cView commit details -
Use regionTimeInfo's region to export template/refcat datasets
This changes the APIs of _export_skymap_and_templates and _export_refcats to take a lsst.sphgeom.Region directly. Note that the centroid of the region is not the same of the detector center, but it should not matter. Because htm7 can be too coarse compared to the patch size, using htm7 indices to search for templates may lead to preloading more patches than necessary and wasting time. This feature of using htm7 to search for overlapping templates is also about to be deprecated and replaced by the arbitrary spatial region query in Butler. The usage will be replaced when switching to the new butler query system.
Configuration menu - View commit details
-
Copy full SHA for 407b263 - Browse repository at this point
Copy the full SHA 407b263View commit details -
Configuration menu - View commit details
-
Copy full SHA for eff9ca6 - Browse repository at this point
Copy the full SHA eff9ca6View commit details -
Query one datasetType at a time for calibs/refcats/templates
This is a preparation step before switching to the new butler query system. In the new query system, query_datasets takes one datasetType at a time, while butler.registry.queryDatasets can take a list of dataset types in the old system. So, we change to query (via _filter_datasets) one type of calibs/refcats/templates at a time. Currently we can preload more types of calibs/refcats/templates than the actual pipelines really need. It's possible that some types are not preloaded but it's okay. For now we allow _MissingDatasetError.
Configuration menu - View commit details
-
Copy full SHA for 057e256 - Browse repository at this point
Copy the full SHA 057e256View commit details -
Use the new Butler query system in _filter_datasets
The new Butler query systems supports spatial-constraint query via lsst.sphgeom.Region directly. With this change, we use it in template and refcat search. This needs stack w_2024_38 or newer. make_export.py uses _filter_datasets so it needs to adjust to the new underlying API too.
Configuration menu - View commit details
-
Copy full SHA for c6860f8 - Browse repository at this point
Copy the full SHA c6860f8View commit details -
Enable prep_butler unit tests again
Some unit tests were temporarily marked expectedFailure in 674a4b1. Now that we switch to the new query system, make them work again. The test repo was put together using middleware tools, which intrinsically uses butler repo's visit-detector regions with its padding from defineVisits config. That padding config is not the same as the preload region padding in prompt processing. This explains the patch differences in template selection.
Configuration menu - View commit details
-
Copy full SHA for 5653980 - Browse repository at this point
Copy the full SHA 5653980View commit details