Percentage Closer Filtering for shadows

[superdump]

Description

• bevy's shadows currently use 'hardware' 2x2 Percentage Closer Filtering resulting in anti-aliased edges of shadows. However, the filtering is insufficient for smooth shadow edges and more extensive/advanced filtering patterns can be used to producer smoother-looking shadows.
  • 'hardware' is in single quotes because apparently in modern GPUs there may not be dedicated silicon for this purpose, but rather the functionality is software in drivers that leverages more generic hardware acceleration functionality.

Background and Reference Material

• Percentage Closer Filtering involves taking multiple samples from the shadow map and comparing with an interpolated mesh surface depth projected into the frame of reference of the light and calculating the percentage of samples in the depth buffer that are closer to the light (i.e. casting shadow) than the mesh surface.
• Sampling from multiple shadow map texels away from the one that 'covers' the fragment on the mesh surface and using the same 'biased' (offset) fragment position easily produces shadow acne/self-shadowing even if some bias configuration works well for sampling texels close to the centre of the kernel.
• Roughly-speaking, more samples are needed to gain more information in order to produce a smoother result. Sampling the same texels repeatedly doesn't help, so more samples roughly means sampling with a larger radius.
• Percentage Closer Soft Shadows (a separate issue) scale both the occluder search radius and the filtering radius based on the distance to the light, and occluder, and the size of the light itself. This can result in large radius sampling kernels which easily produces artifacts due to self-shadowing
• Regular grid sampling patterns produce filtering artifacts like banding.
• Irregular sampling patterns such as blue noise or poisson disc require fewer samples for the same visual quality, don't produce the same banding artifacts, and performance can be scaled nicely (you can choose an arbitrary number of samples to take).
• The Witness has a really nice and smooth PCF sampling and post-filtering (as in, filtering the results of sampling) algorithm (http://the-witness.net/news/2013/09/shadow-mapping-summary-part-1/) that is among the best I have seen for surfaces without much texture (combining textured surfaces and noisy shadows is less visually jarring than noisy shadows on flat shaded surfaces where the noise stands out). I think this would be a good default. It requires relatively few samples to look good too.
• Randomly-rotated (using blue noise for the rotations) poisson disc / direct blue noise sampling would work well with Temporal Anti-Aliasing and would also require few samples, but is visually noisy without it.
• Cubemaps are left-handed!!!
• Biasing
  • Receiver Plane Depth Bias (Isidoro 2006, Page 36) - This can be used for 2D shadow maps and it is quite simple to implement and compute: https://developer.amd.com/wordpress/media/2012/10/Isidoro-ShadowMapping.pdf However, I do not know if it is possible to adapt this to be used for cube shadow maps for point lights so I went in search of alternatives.
  • Adaptive Bias for Shadow Maps (Dou et al 2014) - https://jcgt.org/published/0003/04/08/ This approach is very intuitive and works well. It assumes the mesh surface is locally flat, calculates the gradient of the surface to calculate an 'optimal' fragment depth at the surface at the offset position in light view x/y, and then uses an additional 'adaptive epsilon' that is based on the gradient of the depth projection being used to account for, if I recall correctly, the precision distribution in order to move the fragment position just closer to the light to avoid self-shadowing.
  • Adaptive Bias for Soft Shadows (Ehm et al. 2015) - https://w3-o.cs.hm.edu/users/nischwit/public_html/AdaptiveDepthBias_WSCG.pdf - included here as it details a good further enhancement to Adaptive Bias for Shadow Maps though its extension to Percentage Closer Soft Shadows is not relevant here.

Solution

• https://github.com/superdump/bevy/tree/shadow-pcf
  • A large bunch of experiments to try to get both point and directional light shadow filtering to work well with large filter kernels
  • Implements left-handed cube maps. This was a long-and-hard-fought realisation as documentation for cubemaps is awful. They are left-handed and so we try to force them to work with right-handed projections, rather accept they are left-handed and go with it. It makes for much more comprehensible code that one can reason about!
  • Implements the above-mentioned techniques (receiver plane depth bias, adaptive bias for shadow maps, part of adaptive bias for soft shadows) and some regular-grid and blue noise sampling pattern techniques
• https://github.com/superdump/bevy/tree/poisson-pcf
  • Implements the blue noise random rotated poisson disc and The Witness sampling techniques

Next Steps

• Clean up the conversion of cube map handling to be left-handed and make a PR
• Clean up Adaptive Depth Bias for Shadow Maps with the improvement to scale the adaptive epsilon by min(1 / dot(light dir, frag normal)^2, 100) as introduced in Adaptive Depth Bias for Soft Shadows, make this depth biasing an option, benchmark it, and make a PR
• Clean up the The Witness PCF sampling approach, and make a PR
• Investigate the value of direct blue noise sampling and blue noise random rotated poisson disc sampling and decide whether they are worth integrating now or should wait for reevaluation when we have Temporal Anti-Aliasing

[Suya1671]

Any updates to this? The last commit in that branch was nearly a year ago

[JMS55]

For point lights, I found this: https://kosmonautblog.wordpress.com/2017/03/25/shadow-filtering-for-pointlights/

[superdump]

@JMS55 yeah. I've been thinking that we should probably move to a shadow atlas and only render/update the tiles we need (each tile could be a directional light shadow map cascade, a spot light shadow map, or one face of a point light shadow 'cubemap'). Then we could prioritise which shadow maps to update (e.g. those with some lighting/occluder change that are closest to the camera?) and potentially support shadow mapping for many more spot/point light sources.

[JMS55]

A new biasing technique: "Oriented Depth Bias" section 4.2 http://www.jp.square-enix.com/tech/library/pdf/2023_FFXVIShadowTechPaper.pdf