Merge textures of BatchedMesh

[kzhsw]

Description

Currently in BatchedMesh, there is 2~3 textures per mesh, if rendering a scene with many animated objects in a BatchedMesh could lead to multiple bind/updates on rendering.

three.js/src/objects/BatchedMesh.js

Lines 210 to 215 in 426c39a

	this._matricesTexture = null;
	this._indirectTexture = null;
	this._colorsTexture = null;
	this._initMatricesTexture();
	this._initIndirectTexture();

Solution

Merge them to 1 texture. Since multi draw extension is targeting WebGL2, there is a floatBitsToInt function that can do a reinterpret_cast in shaders, so it can be possible to have int values in a float texture.
A merged BatchedMesh texture could be like this:

int32_t indirectOffset; // offsets in pixels, data must be pixel aligned
int32_t matricesOffset; // there could be extra padding here to ensure 4 pixels of mat4 in the same row
int32_t colorsOffset;
int32_t padding;// to make it a pixel
int32_t indirectData[];
float matricesData[];
float colorsData[];

And when reading from shader, it could be:

uniform highp sampler2D mergedTexture;

vec4 offsets = texelFetch(mergedTexture, ivec2(0,0), 0);
int indirectOffset = floatBitsToInt(offsets.x);
int matricesOffset = floatBitsToInt(offsets.y);
int colorsOffset = floatBitsToInt(offsets.z);
int width = textureSize(mergedTexture, 0).x;

int indirectVectorOffset = gl_DrawID % 4;
int indirectTextureOffset = indirectOffset + gl_DrawID / 4;
int indirectIndex = floatBitsToInt(texelFetch( mergedTexture, ivec2( indirectTextureOffset % size, indirectTextureOffset / size ), 0 )[indirectVectorOffset]);

int matricesTextureOffset = matricesOffset + indirectIndex * 4;
int matX = matricesTextureOffset % size;
int matY = matricesTextureOffset / size;
mat4 batchingMatrix = mat4(texelFetch( mergedTexture, ivec2( matX , matY  ), 0 ),texelFetch( mergedTexture, ivec2( matX +1, matY  ), 0 ),texelFetch( mergedTexture, ivec2( matX+2 , matY  ), 0 ),texelFetch( mergedTexture, ivec2( matX+3 , matY  ), 0 ));

int colorsOffset = colorsOffset + indirectIndex;
int colorX = colorsOffset % size;
int colorY = colorsOffset / size;
vec4 color = texelFetch(mergedTexture, ivec2( colorX , colorY ), 0 );

In this case, only 1 extra texture would be needed to bind/update per frame for BatchedMesh. This might help to reduce the cpu overhead to bind textures.

Alternatives

Keep the indirectTexture and merge matricesTexture and colorsTexture, and place offsets in first pixel of indirectTexture, in this case floatBitsToInt would not be needed.

Additional context

There would be 1 extra texture read per vertex, could add some gpu overhead.
The bitexactness of float textures could be hardware of driver dependant.
floatBitsToInt might not be available if runtime does not have WebGL2 support.
WGSL has a function similar to floatBitsToInt called bitcast.

Further benchmarks are needed, in case of:

1. Scene with a few BatchedMesh including a lot of animated objects.
2. Scene with a lot of BatchedMeshes.
3. High-end and modern GPUs.
4. Low-end or mobile GPUs.

It could benefit more from the partial update proposal.

[gkjohnson]

Are you seeing performance issues related to texture binding? This seems like an extremely microoptimization. I don't think the added code complexity would be worth it and make things like creating the color texture as-needed (and possibly later the matrix texture #29084) more complicated.

if rendering a scene with many animated objects in a BatchedMesh could lead to multiple bind/updates on rendering.

Maybe I'm not understanding but why would many animated objects in a single BatchedMesh lead to too many texture bind updates?

[gkjohnson]

Closing until there's more information or a demonstration of the performance issues. Otherwise this doesn't seem like a worthwhile change.