Questions about ASTC compression in two special cases

[Cyrisub]

1. When using ASTC with mask textures, i found that the edges of rgb color were significantly blocky because of the alpha translucency. i tried to use -a 2 option but astcenc produced identical results. Does -a option need to build from source with some defines? And how should i measure the radius properly?
2. When using ASTC with low resolution textures like under 256 pixels width, it seems that even 4x4 block size would be too blocky and blur textures a lot. Does that means i should use other less-lost compression format or just leave them uncompressed?

[solidpixel]

When using ASTC with mask textures, i found that the edges of rgb color were significantly blocky because of the alpha translucency.

The -a option shouldn't need a special build, but it's not always perfect.

My first step on complex images is to try the -thorough option - it takes longer to compress, but does normally solve a lot of complex block encoding artefacts.

In general for alpha transparency you want to extrude the edge color value in to the transparent region. Filling transparent areas with e.g. black or white tends to be a bad solution (and fails with post-multiplied blending used in GPUs when filtered, even without compression).

Failing that I would suggest tweaking the channel weighting for mask textures using the -cw option to reduce the channel weight assigned to alpha; e.g. -cw 1 1 1 0.5 would reduce the alpha significance by half. The compression can struggle with rapid opaque-to-transparent alpha transitions because the magnitude of the alpha diff tends to be much larger than the size of the color diff, so alpha accuracy dominates in the error metrics.

Can you share an example texture (uncompressed input so I can try to reproduce here)? What is the full command line you are using?

When using ASTC with low resolution textures like under 256 pixels width, it seems that even 4x4 block size would be too blocky and blur textures a lot.

Why does texture resolution matter? A 4x4 block is a 4x4 block, no matter if the image is 2Kx2K or 16x16. Compress everything, unless you have a compelling case where you might not want to that actually show problems. Text font atlases are about the only case where I'd worry.

[Cyrisub]

I just use the example picture you have used in #208 with latest v4.5.0 build. the original picture i download from github is:

and i try astcenc-avx2.exe -cl .\test.png test0.astc 8x8 -thorough to compress and .\astcenc-avx2.exe -dl .\test0.astc test0.png to decompress (using 8x8 to get the block more noticeable) and it produce:

Then i try astcenc-avx2.exe -cl .\test.png test1.astc 8x8 -thorough -a 2 it produce:

Although they did have different hash (both astc files and png files) but i can hardly tell the -a 2 option had any effect in the result.

[solidpixel]

This is a good example of failures that are mostly caused by lack of edge color extrusion into the transparent region (it's a hard edge, with the transparent region RGB values set to black). Preprocessing the texture to extrude the nearest edge color into the the transparent zone helps a lot of reduce the block artefacts because it reduces the magnitude of the color diffs.

The image in this later comment on that thread is a better example of an extruded edge and has significantly lower artefacts around the edges. Worth noting, this image is not perfect example - the black and red streaks in the extrude are outliers that are different to the leaf color and ideally wouldn't be present. Most texture tools can do better.

#208 (comment)

[Cyrisub]

Seems i misunderstood that -a option will add edge color extrusion internally. But i still confused what does -a actually affect results to improve quality if the mask texture is preprocessed.

[solidpixel]

All -a does is adjust the error weighting for the RGB channels (i.e. effective red channel error is r * a not just r). The radius option ensures sensible values under filtering (-a 0 can be used for NEAREST filtering, -a 1 is needed for LINEAR filtering).

Making -a do something sensible with extrudes is a good idea - I'll raise a new issue to add that.

[solidpixel]

Raised #431 to track the auto-extrude optimization.

[Cyrisub]

Very glad to heard that we will have an easier way to prevent edge block artifacts, as changing all mask textures with an ideal extrusion would be a tough work in our pipeline.

Back to my second question, in fact we found many issues about using 8x8 block low-resolution images. They are in fact high level mipmaps, so were generated from a box filter, when we render them in mobile platform, the blocky edge often blend color between different sampling areas (same situation in texture atlas) that's why i'm wonderng whether should apply such compression on those small textures.

[solidpixel]

Back to my second question, in fact we found many issues about using 8x8 block low-resolution images.

8x8 is a low bitrate encoding (2 bits/texel) so will naturally struggle with blocks that contain high frequency changes. Low bitrate encodings are mostly useful for low frequency data sources that lack hard edges, such as light maps, or textures with fewer channels, such as luminance textures.

There is no magic fix unfortunately; if you need to preserve higher frequency edges you will need to use a higher bitrate. I normally recommend a 6x6 block size as the default starting point for LDR color textures and normal maps, and 4x4 for HDR color textures. Reducing below that needs to be considered on a case-by-case basis depending on the type of data being compressed.

[solidpixel]

Also, remember that if you want to go below 6x6, there are non-square sizes you can try which give smaller reductions in bitrate than jumping straight to 8x8 e.g. 8x5 or 8x6.

[Cyrisub]

Low bitrate encodings are mostly useful for low frequency data sources that lack hard edges, such as light maps

In fact the cases we are facing are more challenge as many textures have composite channel like roughness or AO, so the bitrate often cannot fit a satisfying quality. Also we saw many bleeding color issues in unreal's lightmaps using 6x6 block with mipmap resolution reduction and a shadowmask in alpha, which is also annoying.

Seems that -cw option as you mentioned is also apply in internal compression calculatio. I'm wondering maybe we can have an option to control the code bits (or something equal) of every channel, giving more controls of final results in different use cases.

[solidpixel]

That's not how the encoding works unfortunately - the bitstream can skip encoding channels completely (for a set of specific channel orderings) but for the channels that are encoded it always uses the same number of bits per channel.

[solidpixel]

In fact the cases we are facing are more challenge as many textures have composite channel like roughness or AO, so the bitrate often cannot fit a satisfying quality.

Storing a non-correlated packed channel nearly always needs a higher bitrate - I wouldn't expect 8x8 to be a good fit for these.