Skip to content

Commit

Permalink
Switch measurement pairs handling to dask (#590)
Browse files Browse the repository at this point in the history
* Some initial fixes

* Fix measurements_df type check

* I think this works now

* Fixed os.path.isfile patch

* Fixed test_pipeanalysis_load_two_epoch_metrics_dask

* Fixed test__filter_meas_pairs_df

* test_recalc_measurement_pairs_df

* Fix two epoch plotting tests

* Update column names in test_sources

* Tests should all now pass

* Seems to work??

* Initial cleanup

* Manual cleanup

* PEP8
  • Loading branch information
ddobie authored Feb 12, 2025
1 parent fcb18e8 commit a22064a
Show file tree
Hide file tree
Showing 4 changed files with 241 additions and 312 deletions.
2 changes: 1 addition & 1 deletion tests/data/recalc_sources_df_output.csv
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
id,wavg_ra,wavg_dec,avg_compactness,min_snr,max_snr,wavg_uncertainty_ew,wavg_uncertainty_ns,avg_flux_int,avg_flux_peak,max_flux_peak,max_flux_int,min_flux_peak,min_flux_int,min_flux_peak_isl_ratio,min_flux_int_isl_ratio,v_int,v_peak,eta_int,eta_peak,new,new_high_sigma,n_neighbour_dist,vs_significant_max_peak,m_abs_significant_max_peak,vs_significant_max_int,m_abs_significant_max_int,n_measurements,n_selavy,n_forced,n_siblings,n_relations
id,wavg_ra,wavg_dec,avg_compactness,min_snr,max_snr,wavg_uncertainty_ew,wavg_uncertainty_ns,avg_flux_int,avg_flux_peak,max_flux_peak,max_flux_int,min_flux_peak,min_flux_int,min_flux_peak_isl_ratio,min_flux_int_isl_ratio,v_int,v_peak,eta_int,eta_peak,new,new_high_sigma,n_neighbour_dist,vs_abs_significant_max_peak,m_abs_significant_max_peak,vs_abs_significant_max_int,m_abs_significant_max_int,n_measurements,n_selavy,n_forced,n_siblings,n_relations
729,321.9001325179949,-6.193030233233051,1.8073103029326183,276.2080200501253,392.31612903225806,0.00013898019685224127,0.00013898019685224127,217.508,120.70375,126.772,222.104,110.207,213.979,1.0,1.0,0.01738472025034037,0.06053938024395404,16.072133072157158,327.6134309054469,False,0.0,0.0,30.212044,0.13980141,5.8425226,0.03726355,4,4,0,0,1
730,324.1633295343454,-5.040717073358131,1.1167516411885052,47.25,57.471365638766514,0.00014096412317649853,0.00014096412317649853,14.525750000000002,12.972000000000001,13.643,17.672,12.096,12.831,1.0,1.0,0.1481646810260763,0.04956644262980651,15.489511624915242,5.842483557954741,False,0.0,2.529947868662914,4.04786,0.12020669,6.483879,0.3174114,4,4,0,0,0
2251,321.9001325179949,-6.193030233233051,1.8073103029326183,276.2080200501253,392.31612903225806,0.00013898019685224127,0.00013898019685224127,217.508,120.70375,126.772,222.104,110.207,213.979,1.0,1.0,0.01738472025034037,0.06053938024395403,16.072133072157158,327.61343090548564,False,0.0,0.0,30.212044,0.13980141,5.8425226,0.03726355,4,4,0,0,1
2 changes: 1 addition & 1 deletion tests/data/test_sources.csv
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
id,n_meas_forced,n_meas,n_meas_sel,n_sibl,wavg_ra,wavg_dec,avg_compactness,min_snr,max_snr,wavg_uncertainty_ew,wavg_uncertainty_ns,avg_flux_int,avg_flux_peak,max_flux_peak,max_flux_int,min_flux_peak,min_flux_int,min_flux_peak_isl_ratio,min_flux_int_isl_ratio,v_int,v_peak,eta_int,eta_peak,n_rel,new,new_high_sigma,n_neighbour_dist,vs_significant_max_peak,m_abs_significant_max_peak,vs_significant_max_int,m_abs_significant_max_int
id,n_meas_forced,n_meas,n_meas_sel,n_sibl,wavg_ra,wavg_dec,avg_compactness,min_snr,max_snr,wavg_uncertainty_ew,wavg_uncertainty_ns,avg_flux_int,avg_flux_peak,max_flux_peak,max_flux_int,min_flux_peak,min_flux_int,min_flux_peak_isl_ratio,min_flux_int_isl_ratio,v_int,v_peak,eta_int,eta_peak,n_rel,new,new_high_sigma,n_neighbour_dist,vs_abs_significant_max_peak,m_abs_significant_max_peak,vs_abs_significant_max_int,m_abs_significant_max_int
729,0,5,5,1,321.90006200895346,-6.19328120595884,1.6497708211574185,276.2080200501253,392.31612903225806,0.00012430646757473707,0.00012430646757473707,196.96380000000002,119.0788,126.772,222.104,110.207,114.787,0.6058888745371567,0.6098230887743717,0.23382367320174888,0.0612809995805187,3595.588994778409,345.1476926590294,1,False,0.0,0.0006578635631167275,30.21204300309697,0.13980141700319448,93.84061084533053,0.6371022081326008
730,0,5,5,0,324.1633336543154,-5.040746047434953,1.0532970708775455,47.25,57.471365638766514,0.00012597876059013304,0.00012597876059013304,13.829400000000001,13.1404,13.814,17.672,12.096,11.044,1.0,1.0,0.17561736823859211,0.051155411831745705,24.630200983798716,6.862681205039394,0,False,0.0,0.03902594624083344,4.856547003561874,0.13261289077576224,9.635110562912836,0.461624181640897
2251,0,5,5,1,321.90044443824974,-6.192744331356945,1.6464327102481555,183.0725,392.31612903225806,0.00012432638100232205,0.00012432638100232205,188.69500000000002,111.2088,126.772,222.104,73.229,73.443,0.3941111254628434,0.3901769112256282,0.34187973190761733,0.1992146017123999,7204.5820982079385,3235.2523805832607,1,False,0.0,0.0006578635631167572,101.04454706219408,0.5354273228633857,130.73169739380583,1.0060058129502245
Loading

0 comments on commit a22064a

Please sign in to comment.