Integral Image

[rold2007]

I was searching for the possibility of computing an image integral with ImageSharp and I saw that the algorithm exists but it is integrated within AdaptiveThresholdProcessor.

Should this algorithm be exposed in the API, adding support for more/all pixel formats ?

[antonfirsov]

Defining the right output format and API seems to be the hardest task here. AdaptiveThresholdProcessor adds all channels to an UInt64 buffer.
Is this the format you would expect? Can you elaborate on your use case?

[JimBobSquarePants]

As I recall integral images were used initially with mipmaps so if we do go ahead we should also look at the mipmap generation code the the textures library to ensure the design covers any required cases there.

[antonfirsov]

/cc @UkooLabs (Also see my comment below.)

[equinox2k]

@JimBobSquarePants @antonfirsov regarding textures library and as you know I started creating various new pixel formats needed, There are many pixel formats some packed, some byte per channel up to int64 per channel. Then some signed and unsigned. The issues I have been trying to overcome was the signed / unsigned conversion especially when numbers where already Int64. So if a image integral is all the channels added to int64 then it would be only possible for some formats. Also some formats seem to switch between being packed or not when they can be represented both ways (so maybe some standardization needed).

[rold2007]

@antonfirsov The ouput buffer will need to be UInt64 to prevent arithmetic overflow. The code for AdaptiveThresholdProcessor applies the integral imager on an RGB buffer, but usually, in my experience when doing image processing, it is applied to 8-bit grayscale images (so L8 for ImageSharp). For example: Haar-like features. So the input would be one grayscale image and the output would be one UInt64 image.
I don't think it would be useful to provide the code to extract the sum of a rectangle as it would probably be faster for the user to compute many sums inside an unsafe loop instead of calling ImageSharp for each sum.

@JimBobSquarePants I'm not familiar with the mipmap generation code you're talking about and a quick search in the code didn't bring anything. But I agree that if it is useful somewhere else, then better to make it general enough to supports all needed cases.

[JimBobSquarePants]

I'm swaying here.... Can't decide.

The extension method with specific pixel formats look good if we can template them, otherwise it's a chore. I'm not sure I like the idea of accepting all but throwing (though this is an established pattern with Vector<T>) and I can be convinced if we can create a fast generic implementation.

I don't think it's out of scope but If we were to implement it it would be fairly low priority just now and it would likely require an external PR.

[antonfirsov]

Definitely "up for grabs" category. @rold2007 are you interested raising a PR?

My main question was whether we want to let such a feature in or not. The concrete API could be polished out in further discussions / PR then.

[JimBobSquarePants]

So I misremembered Summed Area Tables are an alternative to mipmaps. (I knew they were related somehow)

Here's Frank Crow's original paper on the subject for anyone who is curious.
http://www.florian-oeser.de/wordpress/wp-content/2012/10/crow-1984.pdf

I'm gonna go with yes. Once a concrete API is designed and implemented we can then use that ourselves in AdaptiveThresholdProcessor

[rold2007]

Thanks for your replies @antonfirsov and @JimBobSquarePants. I understand that the feature is interesting for ImageSharp but it is not on the priority list. I'll try to provide a PR, but it will obviously take some time. In terms of design, I also don't like the throwing version.
I've downloaded Crow's paper, I'll read it soon.

[JimBobSquarePants]

Awesome. Thanks!