Fix broken tests in Better Tracing & Skeletonisation Merger branch

[ns-rse]

Whilst working on writing unit tests for maxgamill-sheffield/800-better-tracing which addresses #800 I found that the tests suite does not complete as there are a number of breaking changes.

In total there are 61 tests fail, output with basic details are below...

ERROR tests/test_plotting.py::test_plot_violin - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_set_label[grain_curvature1-Smaller Curvature] - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_plot_violin_multiple_directories - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_set_label_keyerror - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_plot_kde - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_plot_kde_multiple_directories - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_plot_hist - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_plot_hist_multiple_directories - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_plot_hist_kde - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_plot_hist_kde_multiple_directories - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_ordered_trace_heights[dnatrace_linear-118-8.8224769e-10-1.7610771e-09] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_ordered_trace_heights[dnatrace_circular-151-2.5852866e-09-2.5852866e-09] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_gaussian_filter[dnatrace_linear-5.517763534147536e-06] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_ordered_get_trace_cumulative_distances[linear] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_disordered_trace[dnatrace_linear-thin-187-start6-end6] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_gaussian_filter[dnatrace_circular-6.126947266262167e-06] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_ordered_get_trace_cumulative_distances[circular] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_disordered_trace[dnatrace_circular-thin-190-start7-end7] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_disordered_trace[dnatrace_linear-topostats-120-start0-end0] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_linear_or_circular[dnatrace_circular-True] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_fitted_traces[dnatrace_linear-118-start0-end0] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_measure_contour_length[dnatrace_linear-9.040267985905398e-08] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_disordered_trace[dnatrace_circular-topostats-150-start1-end1] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_ordered_traces[dnatrace_linear-118-start0-end0] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_measure_contour_length[dnatrace_circular-7.617314045334366e-08] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_disordered_trace[dnatrace_linear-zhang-170-start2-end2] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_fitted_traces[dnatrace_circular-151-start1-end1] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_ordered_traces[dnatrace_circular-151-start1-end1] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_measure_end_to_end_distance[dnatrace_linear-0] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_disordered_trace[dnatrace_circular-zhang-184-start3-end3] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_splined_traces[dnatrace_linear-1652-start0-end0] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_measure_end_to_end_distance[dnatrace_circular-0] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_splined_traces[dnatrace_circular-2114-start1-end1] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_disordered_trace[dnatrace_linear-lee-130-start4-end4] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/tracing/test_dnatracing_single_grain.py::test_get_disordered_trace[dnatrace_circular-lee-177-start5-end5] - TypeError: dnaTrace.__init__() got an unexpected keyword argument 'skeletonisation_method'
ERROR tests/test_plotting.py::test_df_from_csv - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_toposum_class - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_outfile - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_set_label[contour_length-Contour Length] - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_set_label[end_to_end_distance-End to End Distance] - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
ERROR tests/test_plotting.py::test_set_label[grain_bound_len-Circumference] - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
FAILED tests/test_plotting.py::test_toposum - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
FAILED tests/test_grainstats_minicircle.py::test_grainstats_regression
FAILED tests/test_plottingfuncs.py::test_plot_and_save_non_square_bounding_box - Failed: Error: Image files did not match.
FAILED tests/test_grainstats_minicircle.py::test_cropped_image - assert (1, 1) == (21, 21)
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain1-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain2-True] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain7-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain0-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain3-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain8-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/test_plottingfuncs.py::test_plot_and_save_no_colorbar - Failed: Error: Image files did not match.
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain13-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain9-True] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain4-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain14-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain10-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain5-True] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain11-False] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_get_splined_traces[trace_image0-1.0-True-0.0-expected_spline_image0] - AttributeError: type object 'genTracingFuncs' has no attribute 'countNeighbours'
FAILED tests/tracing/test_dnatracing_multigrain.py::test_trace_image[multigrain_lee-lee-1-statistics2-ordered_trace_start2-ordered_trace_end2] - TypeError: trace_image() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_methods.py::test_get_splined_traces[trace_image3-10.0-True-5.0-expected_spline_image3] - AttributeError: type object 'genTracingFuncs' has no attribute 'countNeighbours'
FAILED tests/tracing/test_dnatracing_multigrain.py::test_trace_image[multigrain_topostats-topostats-1-statistics0-ordered_trace_start0-ordered_trace_end0] - TypeError: trace_image() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/test_plottingfuncs.py::test_mask_cmap - Failed: Error: Image files did not match.
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain6-True] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_methods.py::test_linear_or_circular[grain12-True] - AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'?
FAILED tests/tracing/test_dnatracing_multigrain.py::test_trace_image[multigrain_thin-thin-1-statistics3-ordered_trace_start3-ordered_trace_end3] - TypeError: trace_image() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_multigrain.py::test_trace_image[multigrain_zhang-zhang-1-statistics1-ordered_trace_start1-ordered_trace_end1] - TypeError: trace_image() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_methods.py::test_get_splined_traces[trace_image1-1.0-True-10.0-expected_spline_image1] - AttributeError: type object 'genTracingFuncs' has no attribute 'countNeighbours'
FAILED tests/tracing/test_dnatracing_multigrain.py::test_prep_arrays[0-target_image0-target_mask0] - ValueError: too many values to unpack (expected 2)
FAILED tests/tracing/test_dnatracing_methods.py::test_get_splined_traces[trace_image4-5.0-False-5.0-expected_spline_image4] - AttributeError: type object 'genTracingFuncs' has no attribute 'countNeighbours'
FAILED tests/tracing/test_dnatracing_multigrain.py::test_prep_arrays[1-target_image1-target_mask1] - ValueError: too many values to unpack (expected 2)
FAILED tests/tracing/test_dnatracing_multigrain.py::test_image_trace_unequal_arrays - TypeError: trace_image() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_methods.py::test_get_splined_traces[trace_image2-10.0-True-0.0-expected_spline_image2] - AttributeError: type object 'genTracingFuncs' has no attribute 'countNeighbours'
FAILED tests/tracing/test_dnatracing_methods.py::test_get_splined_traces[trace_image5-4.0-False-5.0-expected_spline_image5] - AttributeError: type object 'genTracingFuncs' has no attribute 'countNeighbours'
FAILED tests/tracing/test_dnatracing_single_grain.py::test_trace_grain[cropped_image0-cropped_mask0-linear_test_topostats-topostats-3.115753758716346e-08-False-5.684734982126664e-08] - TypeError: trace_grain() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_methods.py::test_get_splined_traces[trace_image6-5.0-False-0.0-expected_spline_image6] - AttributeError: type object 'genTracingFuncs' has no attribute 'countNeighbours'
FAILED tests/tracing/test_dnatracing_single_grain.py::test_trace_grain[cropped_image1-cropped_mask1-circular_test_topostats-topostats-0-True-7.617314045334366e-08] - TypeError: trace_grain() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_single_grain.py::test_trace_grain[cropped_image2-cropped_mask2-linear_test_zhang-zhang-2.6964685842539566e-08-False-6.194694383968303e-08] - TypeError: trace_grain() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_single_grain.py::test_trace_grain[cropped_image4-cropped_mask4-linear_test_lee-lee-3.197879765453915e-08-False-5.655032001817721e-08] - TypeError: trace_grain() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_single_grain.py::test_trace_grain[cropped_image3-cropped_mask3-circular_test_zhang-zhang-9.636691058914389e-09-False-8.187508931608563e-08] - TypeError: trace_grain() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_methods.py::test_get_splined_traces[trace_image7-5.0-False-5.0-expected_spline_image7] - AttributeError: type object 'genTracingFuncs' has no attribute 'countNeighbours'
FAILED tests/tracing/test_dnatracing_single_grain.py::test_trace_grain[cropped_image5-cropped_mask5-circular_test_lee-lee-8.261640682714017e-09-False-8.062559919860788e-08] - TypeError: trace_grain() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_dnatracing_single_grain.py::test_trace_grain[cropped_image6-cropped_mask6-linear_test_thin-thin-4.068855894099921e-08-False-5.518856387362746e-08] - TypeError: trace_grain() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_pruning.py::TestTopoStatsPrune::test_find_branch_ends[Linear array with two forks at one end]
FAILED tests/tracing/test_dnatracing_single_grain.py::test_trace_grain[cropped_image7-cropped_mask7-circular_test_thin-thin-3.638262839374549e-08-False-3.6512544238919716e-08] - TypeError: trace_grain() got an unexpected keyword argument 'skeletonisation_method'
FAILED tests/tracing/test_pruning.py::TestTopoStatsPruneMethods::test_prune_skeleton[Pruning by length disabled and leaves branch, height threshold decreased] - AttributeError: 'heightPruning' object has no attribute 'astype'
FAILED tests/tracing/test_pruning.py::TestHeightPruningBasic::test_remove_bridges[ring with branches] - AssertionError: 
FAILED tests/test_plottingfuncs.py::test_plot_and_save_colorbar_and_axes - Failed: Error: Image files did not match.
FAILED tests/test_plottingfuncs.py::test_plot_and_save_no_axes - Failed: Error: Image files did not match.
FAILED tests/test_plottingfuncs.py::test_plot_and_save_no_axes_no_colorbar - Failed: Error: Image files did not match.
FAILED tests/test_plottingfuncs.py::test_high_dpi - Failed: Error: Image files did not match.
FAILED tests/test_plottingfuncs.py::test_mask_dilation - Failed: Error: Image files did not match.
FAILED tests/test_plotting.py::test_melt_data - KeyError: "The following id_vars or value_vars are not present in the DataFrame: ['grain_number']"
FAILED tests/test_processing.py::test_process_scan_below - IndexError: list index out of range
FAILED tests/test_processing.py::test_process_stages[Filtering, Grain, GrainStats and DNA Tracing enabled] - KeyError: "'green_black' is not a known colormap name"
FAILED tests/test_processing.py::test_process_scan_no_grains - KeyError: "'green_black' is not a known colormap name"
FAILED tests/test_processing.py::test_process_scan_above - KeyError: "'green_black' is not a known colormap name"
FAILED tests/test_processing.py::test_process_scan_align_grainstats_dnatracing - IndexError: list index out of range
FAILED tests/test_processing.py::test_process_scan_both - KeyError: "'green_black' is not a known colormap name"
FAILED tests/test_processing.py::test_run_dnatracing - TypeError: run_dnatracing() got an unexpected keyword argument 'grain_out_path'
FAILED tests/test_run_topostats.py::test_run_topostats_process_debug
FAILED tests/test_run_topostats.py::test_run_topostats_process_all

Many of these are unexpected keyword argument which suggests one of a few things...

1. New arguments have been added to the configuration file but the existing methods have not been updated to accept these.
2. Arguments have changed name.

Common keyword arguments that crop up repeatedly in errors and possible causes/problems and other types of errors are listed below.

• skeletonisation_method - which is a new argument introduced in this branch but not updated in many test files (by far the most common cause of errors/failures).
• countNeighbours - looks like an old function of the genTracingFuncs module that has been moved/renamed/removed.
• AttributeError: 'dnaTrace' object has no attribute 'mol_is_circular'. Did you mean: 'mol_is_circulars'? - looks like a renaming issue where plural is now used, should be easy to correct.
• KeyError : 'green_black' is not a known colormap name - a new colormap introduced in some manner, probably needs to be defined in topostats/theme.py although it does have a new blu_purp_green colormap included so perhaps what is required is green_black > blu_purp_green (personally I would favor not using abbreviations tab-auto-complete saves typing and full names are easier to remember than abbreviations so blue_purple_green).
• IndexError - crops up in test_processing.py::test_process_scan_align_grainstats_dnatracing
• A number of test_plottingfuncs.py do not match the existing images this will need investigating and resolving.