Incorporating height profile into GrainStats

[ns-rse]

Closes #748

After the protracted wrestling I had of how to get minimum feret distances out (see #755) I've decided to finish this work off now as I'm unlikely to look at it again until September and had been tinkering with it over the last week and it is more efficient to finish it off now than struggle to remember where I was in two months time.

This PR includes two commits, importantly one fixes a problem with the topostats.measure.height_profiles.interpolate_height_profiles() which was introduced in #859 (details on that below).

Height Profiles

The option extract_height_profile has been introduced to the grainstats section of default_config.yaml, when true this will trigger extraction of height profiles along the maximum feret. These are aggregated across all "grains" and images and written to a JSON file (height_profiles.json) adjacent to all_statistics.csv in the specified output directory.

JSON is used because CSV is not particularly conducive to this data structure as profiles are taken across cropped images which differ in size.

This necessitated adding a dependency for numpyencoder which correctly and safely converts Numpy dtypes to a format that json.dump() can write.

I have a suspicion that at some point it will be desirable to generate line plots of these which is possible but for now users will have to extract the data align and plot these themselves (if anyone does this and wants to share or contribute their solution to TopoStats that would be very much appreciated).

Fixing interpolate_height_profiles() #859

In #859 I had mistakenly used np.sort() on the feret co-ordinates. Two of tests developed there passed, one I opted to skip (The maximum feret was almost horizontal and this should have raised flags as it was indicative of another problem, namely that we should interpolate in such a manner as to get as many points as possible along the line of the maximum feret)

When it came to developing tests to incorporate height profile extraction into the work flow of GrainStats() it revealed that things were not as they should be as whilst the feret coordinates were correct point interpolation resulted in...

Clearly not what we wanted! This arose because np.interp() expects "monotonically increasing sample points" and in some of the above the feret coordinates were returned with the highest value first and were therefore decreasing.

Solution

In order to get the interpolation correct after identifying these problems two changes were made.

1. Assess which is the largest distance between the feret co-ordinates on the x and y axis and the largest has points linearly interpolated first.
2. Order the feret points in ascending order (depending on whether it is the x or y axis that is the largest distance as this is the one along which interpolation will occur).

The points are now correctly interpolated regardless of the order in which the feret coordinates are returned and we get the greatest detail by doing so along the longest x/y axis first.

As a bonus the test that was previously skipped is now looking as expected and has been reinstated.

[ns-rse]

Ready for review as I figured out how to update the HDF5 (see #873 for details and future reference).

In fixing this I found there was a precision error in the heights on OSX compared to Linux (where the tests were developed) to which end I split the regression test of heights out to their own functions and skip them if the reported platform.platform()[:5] == "macOS". This adds a slight increased overhead as in essence two of the tests are run twice but assessed differently but I couldn't see an option for format float's when using json.dump().

I also noticed a missing f-string in one of the plottingfuncs.py LOGGER.info() so updated that.

[SylviaWhittle]

This works for me, and I am able to extract the height profiles from the HDF5 file and plot them.

Thank you very much for your work on this 🎉

[SylviaWhittle]

Here is how I plotted the heights in case anyone reads this

[ns-rse]

Here is how I plotted the heights in case anyone reads this

It should be possible to iterate over the dictionary directly using .items(). Following is untested but should achieve the same thing.

from pathlib import Path
from loguru import logger
import numpy as np
import h5py
import matplotlib.pyplot as plt

FILE_PATH = Path("./minicircle.topostats")
with h5py.File(FILE_PATH, "r") as f:
    # print keys
    logger.info(f.keys())
    height_profiles = f["height_profiles"]["above"]

fig, ax = plt.subplots(figsize=(10,5))
for key, profile in height_profiles.items()
    ax.plot(profile, label=key)
ax.set_xlabel("Pixel")
ax.set_ylabel("Height (nm)")
ax.set_title("Height Profiles")
plt.legend()
plt.show()

[SylviaWhittle]

🎉