Reading a 2-dimensional array of structs with HDF5-CSharp

[flappah]

Hi Lior, I got a bit stuck and was wondering if you could give me a pointer in the right direction. I am faced with an HDF5 file that has, at its core, a 2 dimensional array of structs. Each struct consists of two floats that represent the surface-current-speed and the surface-current-direction. Now what I did was try the Hdf5.ReadCompounds function and give it the following struct as T:

using HDF5CSharp.DataTypes;

namespace S1XViewer.HDF
{
    public struct SurfaceCurrentInstance
    {
        [Hdf5EntryName("surfaceCurrentSpeed")]
        public double speed;

        [Hdf5EntryName("surfaceCurrentDirection")]
        public double direction;

        /// <summary>
        ///     
        /// </summary>
        /// <param name="height"></param>
        /// <param name="trend"></param>
        public SurfaceCurrentInstance(double speed, double direction)
        {
            this.speed = speed;
            this.direction = direction;
        }
    }
}

I've added the HDF5 file for your convenience so you get an idea what I'm trying to read. The relevant dataset is present in each /SurfaceCurrent/SurfaceCurrent.01/Group_0xx group and is called 'values'. When you open it you see an array of 596x433 of structs that has a surfaceCurrentSpeed and a surfaceCurrentDirection.

The closest I got was using the ReadCompounds function as you can see in file:

https://github.com/flappah/S1XViewer/blob/master/S1XViewer.Model/S111DCF2DataParser.cs

It calls on:

https://github.com/flappah/S1XViewer/blob/master/S1XViewer.HDF/DatasetReader.cs

which tries to read the compounds dataset. It does read it but it returns an array of 596 SurfaceCurrentInstances and not a 2 dimensional array. I tried a couple of ways to tackle this but somehow I'm not getting it right. Could you give me a pointer in the right direction? Thanks in advance!

111US00_CBOFS_20221013T00Z.h5.zip

[flappah]

Not sure if you've already seen this question but I did a quick scan of the PureHDF library and that one is able to read multidimensional arrays. I'm not sure if it's also able to read 2d arrays of structs but I could trick it into believing I want 2d arrays of floats. And then it returns that array nicely. I can then parse that back into SurfaceCurrents since the layout is fixed. But I'm still curious if HDF5-CSharp is able to do 2d arrays of structs!

[LiorBanai]

Hi @flappah I wasn't able to check it yet.. (busy with real life stuff unfortunately )..
Can you share the code you used to interact with PureHDF library so I can try later to see similarities?

BTW, pure HDF is great library for reading hdf5 files so if you only need reading that one is better

[flappah]

Hey Lior!

Yeah, no worries. I was a bit stumped on Monday not being able to get things done so I fired off the question. Today I had a fresh start and was reading the PureHDF documentation and there was this mention of multidimensional arrays. That got me going! :)

The actual method for reading 2 dimensional arrays of doubles is currently this one:

        public float[,] ReadArrayOfFloats(string fileName, string name, int dim1, int dim2)
        {
            var mmf = MemoryMappedFile.CreateFromFile(fileName);
            var accessor = mmf.CreateViewAccessor();
            var file = H5File.Open(accessor);

            if (file != null)
            {
                try
                {
                    var dataset = file.Dataset(name);
                    var data2d = dataset.Read<float>().ToArray2D<float>(dim1, dim2 * 2);
                    return data2d;
                }
                catch { }
                finally
                {
                    mmf.Dispose();
                }
            }

            return new float[0, 0]; 
        }

It's in the DatasetReader class: https://github.com/flappah/S1XViewer/blob/master/S1XViewer.HDF/DatasetReader.cs

In the class S111DCF2DataParser.cs at https://github.com/flappah/S1XViewer/blob/master/S1XViewer.Model/S111DCF2DataParser.cs it's called on as:

float[,] speedAndDirectionValues =
    _datasetReader.ReadArrayOfFloats(hdf5FileName, minGroup.Children[0].Name, numPointsLatitude, numPointsLongitude);

[edit]
Oh and while posting this I spotted a bug which I'm fixing right now. It's the dim2 size multiplied by two. While that works great in my current usage, another array read may have different requirements. I'm changing that as we speak.