Add basic smoke tests for topology branch

[SylviaWhittle]

This PR adds basic smoke tests for disordered tracing, node stats, ordered tracing and splining.

[SylviaWhittle]

Pre-commit problems list (shown below)

I believe that the files and code listed are not code that I have touched. Correct me if I've not noticed something 😄

[ns-rse]

I believe that the files and code listed are not code that I have touched.

Nope, none of those are in the files you've touched.

We should all be using the pre-commit configuration that is part of the distribution so that the CI checks pass (yes the target branch here isn't main but eventually it will be being merged into main and will have to pass all these checks, so its better practice to use these checks and get things right in the first instance).

Within the TopoStats directory run the following to install...

pre-commit install

It will then highlight all these problems before changes can be pushed.

[ns-rse]

Bunch of comments in-line, great work writing all these tests @SylviaWhittle 👍

The tests/resources/ is starting to look somewhat cluttered. I think many of the .csv files could be removed if we used pytest-regtest (some comments in-line on this).

That still leaves a lot of files though, many of the filenames carry common suffixes and its clear that many of the objects (.pkl and .npy) are related. I can think of two options to improve organisation...

1. Bundle similar objects into a dictionary with the keys formed from the component that distinguishes them and save as a single pickle.

example_rep_int_all_images.pkl
example_rep_int_all_images_nodestats.pkl
example_rep_int_disordered_crop_data.pkl
example_rep_int_disordered_tracing_stats.csv
example_rep_int_grainstats_additions_df.csv
example_rep_int_grainstats_additions_nodestats.csv
example_rep_int_labelled_grain_mask_thresholded.npy
example_rep_int_nodestats_branch_images.pkl
example_rep_int_nodestats_data.pkl
example_rep_int_ordered_tracing_data.pkl
example_rep_int_ordered_tracing_full_images.pkl
example_rep_int_ordered_tracing_grainstats_additions.csv
example_rep_int_splining_data.pkl
example_rep_int_splining_grainstats_additions.csv
example_rep_int_splining_molstats.csv

Would go into a dictionary with keys of...

{
"all_images": <object>
"all_images_nodestats": <object>
"disordered_crop_data": <object>
"disordered_tracing_stats": <object>
"grainstats_additions_df": <object>
"grainstats_additions_nodestats": <object>
"labelled_grain_mask_thresholded": <object>
"nodestats_branch_images": <object>
"nodestats_data": <object>
"ordered_tracing_data": <object>
"ordered_tracing_full_images": <object>
"ordered_tracing_grainstats_additions": <object>
"splining_data": <object>
"splining_grainstats_additions": <object>
"splining_molstats": <object>
}

...and that could be saved as tests/resources/example_rep_int.pkl.

2. Alternatively create a nested directory structure under tests/resources reflecting the common prefixes...

tests/resources/node/
tests/resources/catenanes/
tests/resources/example_rep_int/

...drop the prefixes from the filenames.

[ns-rse]

This is presumably to update the test files when the underlying code changes?

The syrupy package which is compatible with pytest might be an alternative to this. Not used it yet, only became aware of it ar RSECon2024, but its similar to pytest-regtest I think.

[ns-rse]

If the arrays and dictionaries aren't too large I'd be inclined to use the pytest-regtest approach to comparing these.

[ns-rse]

Two things...

1. Could we use DeepDiff? It seems a very general solution. dict_almost_equal() as it stands only compares floats via np.allclose().

2. I see that this is used in a some of the tests introduced but we should probably have a tests/test_io.py::test_dict_almost_equal() to check that it behaves as expected.

[SylviaWhittle]

1 - Upon a quick glance, it DeepDiff seems like it might do what is needed

2 - When I wrote dict_almost_equal, I did add a test in test_io.py, line 172 in this branch.

[SylviaWhittle]

This function was already in the codebase, I just had to move it since I could not import it in the tests/tracing/test_*.py files since the function was in tests/test_io.py and pylint would not let me do from ..test_io.py import dict_almost_equal and python wouldn't recognise from tests/test_io.py import dict_almost_equal so I have moved it to io.py to allow importing in all tests.

[ns-rse]

Self referential! Test code testing itself!

[ns-rse]

Can the Debugging section here be removed? If not would it be useful to have this as a function that we can use to plot co-ordinates?

[SylviaWhittle]

I'd be inclined to keep it as-is if possible because:

1 - I really think it's useful since we are going to be updating this test a lot over the next few weeks probably. I'd only end up re-writing it each time I update the test, slowing down test updates.

2 - I can't think of anywhere else in our codebase that we would need to debug viewing the splines like this and it would take some time to make a function out of it and test it.

That being said it is a lot of code mess and I very much empathise with wanting to get rid of commented-out code since it does tend to linger infinitely

[ns-rse]

Perhaps a middle ground of making it a function within the test?

It would then be possible to have a single line which calls the function that can be commented/uncomment the call to the function for debugging and the test file doesn't have a large amount of lingering code.

I'd did similar with a plotting function I'd written for the topostats.measure.feret.plot_feret()](

TopoStats/topostats/measure/feret.py

Line 430 in 70d02bd

def plot_feret( # pylint: disable=too-many-arguments,too-many-locals # noqa: C901

) which I used for debugging. I already had images that I'd looked at during the debugging process so tests were relatively straight-forward to add.

[ns-rse]

I think perhaps we should use pytest-regtest to compare the statistics that are produced rather than having to have code to update the CSV files when methods change and then read them in to pd.DataFrame() and pd.testing.assert_frame_equal().

We use this approach for other CSVs that the pipeline produces so should be ok here unless there is a specific reason for this approach?

[SylviaWhittle]

Very happy to have pushback on this, but my thinking was trying to keep it all the same style of test with assertions and variables loaded explicitly since when the test fails, I'm finding it easier to debug using a debugger & IDE tools? I tried debugging tests that use pytest-regtest and it's rather difficult with large objects. Perhaps I went about it wrong?

I think we are anticipating quite a bit of iteration and so making it as smooth as possible to view what exactly changed and if it's valid and then to easily update the values is useful.

IIRC pytest-regtest has the excellent override method of pytest --regtest-reset which makes updating tests a dream but I always have to add code and dig to see if the change is legitimate. Do you have a good way of inspecting changes in regtests?

[ns-rse]

When I've had to update tests from pytest-regtest it is in essence a diff which is what many of the assert show (in one form or another, whether that is default Python / pytest / np.testing.assert* / pd.testing.assert*).

Diffs can be tricky to use and understand at times particularly when its a few numbers in the middle of a large CSV and perhaps Numpy/Pandas help with this but there are some alternatives and features that make it easier.

I find the specific changes within a line that Git can be configured to show really useful, one tool for this is delta but personally I use difftastic as it also understands structure of different programming languages and Git can be easily configured to work with it.

The --regtest-reset is in my view a lot quicker than having to uncomment a bunch of lines to write a CSV or pkl out.

Perhaps we should look at how syrupy compares to pytest-regtests and the manual approach?

Broadly I think it is useful to be consistent across a project though, pick one approach and stick with it, it reduces cognitive overhead and makes it easier for others to get to grips with how the repository works (this is true of more than just tests, e.g. always using pathlib rather than mixing it up with os modules).

[SylviaWhittle]

I believe that the files and code listed are not code that I have touched.

Nope, none of those are in the files you've touched.

We should all be using the pre-commit configuration that is part of the distribution so that the CI checks pass (yes the target branch here isn't main but eventually it will be being merged into main and will have to pass all these checks, so its better practice to use these checks and get things right in the first instance).

Within the TopoStats directory run the following to install...

pre-commit install

It will then highlight all these problems before changes can be pushed.

I do have pre-commit, I was going to tidy it up in a follow-up PR that was just focused on that, keeping this PR self-contained but am happy to fix it in this PR 👍

[SylviaWhittle]

DeepDiff doesn't seem to be a viable alternative currently. It cannot handle np.nan values properly.