feat: depth-buffer clipping for retained-mode scenes (GPU-CLIP-002d)

[kolkov]

GPU-CLIP-002d: Depth-buffer clipping for retained-mode scenes

Summary

Use depth buffer for clip region management in retained-mode rendering (scene graph, UI widget tree). Clip shapes are rendered as depth-only draws, subsequent content is rejected by depth test.

Rationale

Correction: gogpu/ui IS retained-mode (widget tree with draw phase). The scene/ package in gg is also retained-mode (scene graph with tile-parallel rendering). Depth-buffer clipping is relevant NOW, not just for hypothetical future games.

Research shows this is the direction Skia Graphite and Flutter/Impeller are moving:

• Skia Graphite: replaced stencil with depth buffer clipping — clip shapes are depth-only draws, decouples shader programs from clip state entirely
• Flutter/Impeller: evolving from stencil to depth-buffer (clip_depth + kDepthEpsilon)

See: NON-RECTANGULAR-CLIPPING-ENTERPRISE-RESEARCH.md §2 (Graphite), §4 (Impeller)

Advantages over stencil

Aspect	Stencil (Phase 2)	Depth buffer (Phase 4)
Nested clips	8-bit limit (255 levels)	24-bit (16M levels)
Clip + z-ordering	Separate concerns	Unified in one buffer
Shader coupling	Stencil test state	Decoupled (depth test only)
Scene graph fit	Moderate	Excellent (ordered draw list)
Complex scenes (games)	Expensive rebinds	Natural ordering

Architecture

scene.Renderer / UI draw phase:
  1. Assign monotonic depth values to clip regions
  2. Render clip shapes as depth-only (color write mask = 0)
  3. Subsequent draws use depth test (GL_LESS_EQUAL)
  4. Depth values encode clip hierarchy (parent < child < content)

Key challenges

• Requires ordered draw submission (natural for retained-mode, awkward for immediate-mode)
• Depth precision management (24-bit float, kDepthEpsilon increments)
• Integration with existing z-ordering in scene graph
• Must coexist with stencil (Phase 2) for immediate-mode API

Dependencies

• Research: study Skia Graphite ClipStack::clipShader() and Impeller clip_depth in detail
• Prerequisite: Phase 2 (feat: stencil-based arbitrary path clipping (GPU-CLIP-002b) #205) should be done first to understand stencil limitations in practice
• scene/ package must support clip commands

Scope

• scene/ retained-mode renderer
• gogpu/ui widget tree rendering
• NOT for immediate-mode dc.Clip() API (stencil is better there)