Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make lh5 types and exceptions picklable #129

Merged
merged 1 commit into from
Jan 14, 2025
Merged

Conversation

ManuelHu
Copy link
Contributor

@ManuelHu ManuelHu commented Jan 13, 2025

For reboost, I heavily use multiprocessing to produce optical maps. While testing things, I became aware of muliple issues concerning the usage of lgdos in a muti-process context.

  • probably uncontroversial: make the esxeptions picklable (i.e. to propagate errors over process boundaries)
  • it is quite useful to be able to transfer small LGDO objects (in my case, lgdo.types.Histogram.Axes) between processes. So I made them all picklable....
    This is partly based on @iguinn's work on the multiprocessing-capable LH5Iterator, and is extended to also support histogram pickling.
  • I also added tests for many LGDO types, to see if pickling works.

Copy link

codecov bot commented Jan 14, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.77%. Comparing base (f5ed430) to head (ef99e39).
Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #129      +/-   ##
==========================================
+ Coverage   78.65%   78.77%   +0.12%     
==========================================
  Files          46       46              
  Lines        3406     3421      +15     
==========================================
+ Hits         2679     2695      +16     
+ Misses        727      726       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gipert
Copy link
Member

gipert commented Jan 14, 2025

Seems uncontroversial to me, merging.

@gipert gipert added enhancement New feature or request types LGDO types labels Jan 14, 2025
@gipert gipert merged commit 269111a into legend-exp:main Jan 14, 2025
15 checks passed
@ManuelHu ManuelHu deleted the pickle branch January 14, 2025 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request types LGDO types
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants