Skip to content

Commit ac079c1

Browse files
committed
[encoding] BumpEstimator utility
Several vello stages dynamically bump allocate intermediate data structures. Due to graphics API limitations the backing memory for these data structures must have been allocated at the time of command submission even though the precise memory requirements are unknown. Vello currently works around this issue in two ways (see #366): 1. Vello currently prescribes a mechanism in which allocation failures get detected by fencing back to the CPU. The client responds to this event by creating larger GPU buffers using the bump allocator state obtained via read-back. The client has the choice of dropping skipping a frame or submitting the fine stage only after any allocations failures get resolved. 2. The encoding crate hard-codes the buffers to be large enough to be able to render paris-30k, making it unlikely for simple scenes to under-allocate. This comes at the cost of a fixed memory watermark of >50MB. There may be situations when neither of these solutions are desirable while the cost of additional CPU-side pre-processing is not considered prohibitive for performance. It may also be acceptable to pay the cost of generally allocating more than what's required in order to make the this problem go away entirely (except perhaps for OOM situations). In that spirit, this change introduces the beginnings of a heuristic-based conservative memory estimation utility. It currently estimates only the LineSoup buffer (which contains the curve flattening output) within a factor of 1.1x-3.3x on the Vello test scenes (paris-30k is estimated at 1.5x the actual requirement). - Curves are estimated using Wang's formula which is fast to evaluate but produces a less optimal result than Vello's analytic approach. The overestimation is more pronounced with increased curvature variation. - Explicit lines (such as line-tos) get estimated precisely - Only the LineSoup buffer is supported. - A BumpEstimator is integrated with the Scene API (gated by a feature flag) but the results are currently unused. Glyph runs are not supported as the estimator is not yet aware of the path data stored in glyph cache.
1 parent 3b17341 commit ac079c1

File tree

6 files changed

+423
-16
lines changed

6 files changed

+423
-16
lines changed

Cargo.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,11 @@ repository.workspace = true
4343

4444
[features]
4545
default = ["wgpu"]
46+
# Enables GPU memory usage estimation. This performs additional computations
47+
# in order to estimate the minimum required allocations for buffers backing
48+
# bump-allocated GPU memory.
49+
# TODO: Turn this into a runtime option used at resolve time and remove the feature.
50+
bump_estimate = ["vello_encoding/bump_estimate"]
4651
hot_reload = []
4752
buffer_labels = []
4853

crates/encoding/Cargo.toml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,16 @@ repository.workspace = true
99

1010
[features]
1111
default = ["full"]
12+
1213
# Enables support for the full pipeline including late-bound
1314
# resources (gradients, images and glyph runs)
1415
full = ["skrifa", "guillotiere"]
1516

17+
# Enables an optional GPU memory usage estimation utility. This can be used to
18+
# perform additional computations in order to estimate the minimum required allocations
19+
# for buffers backing bump-allocated GPU memory.
20+
bump_estimate = []
21+
1622
[lints]
1723
workspace = true
1824

crates/encoding/src/config.rs

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,73 @@ pub struct BumpAllocators {
3737
pub lines: u32,
3838
}
3939

40+
#[derive(Default)]
41+
pub struct BumpAllocatorMemory {
42+
pub total: u32,
43+
pub binning: BufferSize<u32>,
44+
pub ptcl: BufferSize<u32>,
45+
pub tile: BufferSize<Tile>,
46+
pub seg_counts: BufferSize<SegmentCount>,
47+
pub segments: BufferSize<PathSegment>,
48+
pub lines: BufferSize<LineSoup>,
49+
}
50+
51+
impl BumpAllocators {
52+
pub fn memory(&self) -> BumpAllocatorMemory {
53+
let binning = BufferSize::new(self.binning);
54+
let ptcl = BufferSize::new(self.ptcl);
55+
let tile = BufferSize::new(self.tile);
56+
let seg_counts = BufferSize::new(self.seg_counts);
57+
let segments = BufferSize::new(self.segments);
58+
let lines = BufferSize::new(self.lines);
59+
BumpAllocatorMemory {
60+
total: binning.size_in_bytes()
61+
+ ptcl.size_in_bytes()
62+
+ tile.size_in_bytes()
63+
+ seg_counts.size_in_bytes()
64+
+ segments.size_in_bytes()
65+
+ lines.size_in_bytes(),
66+
binning,
67+
ptcl,
68+
tile,
69+
seg_counts,
70+
segments,
71+
lines,
72+
}
73+
}
74+
}
75+
76+
impl std::fmt::Display for BumpAllocatorMemory {
77+
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
78+
write!(
79+
f,
80+
"\n \
81+
\tTotal:\t\t\t{} bytes ({:.2} KB | {:.2} MB)\n\
82+
\tBinning\t\t\t{} elements ({} bytes)\n\
83+
\tPTCL\t\t\t{} elements ({} bytes)\n\
84+
\tTile:\t\t\t{} elements ({} bytes)\n\
85+
\tSegment Counts:\t\t{} elements ({} bytes)\n\
86+
\tSegments:\t\t{} elements ({} bytes)\n\
87+
\tLines:\t\t\t{} elements ({} bytes)",
88+
self.total,
89+
self.total as f32 / (1 << 10) as f32,
90+
self.total as f32 / (1 << 20) as f32,
91+
self.binning.len(),
92+
self.binning.size_in_bytes(),
93+
self.ptcl.len(),
94+
self.ptcl.size_in_bytes(),
95+
self.tile.len(),
96+
self.tile.size_in_bytes(),
97+
self.seg_counts.len(),
98+
self.seg_counts.size_in_bytes(),
99+
self.segments.len(),
100+
self.segments.size_in_bytes(),
101+
self.lines.len(),
102+
self.lines.size_in_bytes()
103+
)
104+
}
105+
}
106+
40107
/// Storage of indirect dispatch size values.
41108
///
42109
/// The original plan was to reuse [`BumpAllocators`], but the WebGPU compatible

crates/encoding/src/estimate.rs

Lines changed: 291 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,291 @@
1+
// Copyright 2024 the Vello Authors
2+
// SPDX-License-Identifier: Apache-2.0 OR MIT
3+
4+
//! This utility provides conservative size estimation for buffer allocations backing
5+
//! GPU bump memory. This estimate relies on heuristics and naturally overestimates.
6+
7+
use super::{BufferSize, BumpAllocatorMemory, Transform};
8+
use peniko::kurbo::{Cap, Join, PathEl, Stroke, Vec2};
9+
10+
const RSQRT_OF_TOL: f64 = 2.2360679775; // tol = 0.2
11+
12+
#[derive(Clone, Default)]
13+
pub struct BumpEstimator {
14+
// TODO: support binning
15+
// TODO: support ptcl
16+
// TODO: support tile
17+
// TODO: support segment counts
18+
// TODO: support segments
19+
lines: LineSoup,
20+
}
21+
22+
impl BumpEstimator {
23+
pub fn new() -> Self {
24+
Self::default()
25+
}
26+
27+
pub fn reset(&mut self) {
28+
*self = Self::default();
29+
}
30+
31+
/// Combine the counts of this estimator with `other` after applying an optional `transform`.
32+
pub fn append(&mut self, other: &Self, transform: Option<&Transform>) {
33+
self.lines.add(&other.lines, transform_scale(transform));
34+
}
35+
36+
pub fn count_path(
37+
&mut self,
38+
path: impl Iterator<Item = PathEl>,
39+
t: &Transform,
40+
stroke: Option<&Stroke>,
41+
) {
42+
let mut caps = 1;
43+
let mut joins: u32 = 0;
44+
let mut lineto_lines = 0;
45+
let mut fill_close_lines = 1;
46+
let mut curve_lines = 0;
47+
let mut curve_count = 0;
48+
49+
// Track the path state to correctly count empty paths and close joins.
50+
let mut first_pt = None;
51+
let mut last_pt = None;
52+
for el in path {
53+
match el {
54+
PathEl::MoveTo(p0) => {
55+
first_pt = Some(p0);
56+
if last_pt.is_none() {
57+
continue;
58+
}
59+
caps += 1;
60+
joins = joins.saturating_sub(1);
61+
last_pt = None;
62+
fill_close_lines += 1;
63+
}
64+
PathEl::ClosePath => {
65+
if last_pt.is_some() {
66+
joins += 1;
67+
lineto_lines += 1;
68+
}
69+
last_pt = first_pt;
70+
}
71+
PathEl::LineTo(p0) => {
72+
last_pt = Some(p0);
73+
joins += 1;
74+
lineto_lines += 1;
75+
}
76+
PathEl::QuadTo(p1, p2) => {
77+
let Some(p0) = last_pt.or(first_pt) else {
78+
continue;
79+
};
80+
curve_count += 1;
81+
curve_lines +=
82+
wang::quadratic(RSQRT_OF_TOL, p0.to_vec2(), p1.to_vec2(), p2.to_vec2(), t);
83+
last_pt = Some(p2);
84+
joins += 1;
85+
}
86+
PathEl::CurveTo(p1, p2, p3) => {
87+
let Some(p0) = last_pt.or(first_pt) else {
88+
continue;
89+
};
90+
curve_count += 1;
91+
curve_lines += wang::cubic(
92+
RSQRT_OF_TOL,
93+
p0.to_vec2(),
94+
p1.to_vec2(),
95+
p2.to_vec2(),
96+
p3.to_vec2(),
97+
t,
98+
);
99+
last_pt = Some(p3);
100+
joins += 1;
101+
}
102+
}
103+
}
104+
let Some(style) = stroke else {
105+
self.lines.linetos += lineto_lines + fill_close_lines;
106+
self.lines.curves += curve_lines;
107+
self.lines.curve_count += curve_count;
108+
return;
109+
};
110+
111+
// For strokes, double-count the lines to estimate offset curves.
112+
self.lines.linetos += 2 * lineto_lines;
113+
self.lines.curves += 2 * curve_lines;
114+
self.lines.curve_count += 2 * curve_count;
115+
116+
let round_scale = transform_scale(Some(t));
117+
let width = style.width as f32;
118+
self.count_stroke_caps(style.start_cap, width, caps, round_scale);
119+
self.count_stroke_caps(style.end_cap, width, caps, round_scale);
120+
self.count_stroke_joins(style.join, width, joins, round_scale);
121+
}
122+
123+
/// Produce the final total, applying an optional transform to all content.
124+
pub fn tally(&self, transform: Option<&Transform>) -> BumpAllocatorMemory {
125+
let scale = transform_scale(transform);
126+
let binning = BufferSize::new(0);
127+
let ptcl = BufferSize::new(0);
128+
let tile = BufferSize::new(0);
129+
let seg_counts = BufferSize::new(0);
130+
let segments = BufferSize::new(0);
131+
let lines = BufferSize::new(self.lines.tally(scale));
132+
BumpAllocatorMemory {
133+
total: binning.size_in_bytes()
134+
+ ptcl.size_in_bytes()
135+
+ tile.size_in_bytes()
136+
+ seg_counts.size_in_bytes()
137+
+ lines.size_in_bytes(),
138+
binning,
139+
ptcl,
140+
tile,
141+
seg_counts,
142+
segments,
143+
lines,
144+
}
145+
}
146+
147+
fn count_stroke_caps(&mut self, style: Cap, width: f32, count: u32, scale: f32) {
148+
match style {
149+
Cap::Butt => self.lines.linetos += count,
150+
Cap::Square => self.lines.linetos += 3 * count,
151+
Cap::Round => {
152+
self.lines.curves += count * estimate_arc_lines(width, scale);
153+
self.lines.curve_count += 1;
154+
}
155+
}
156+
}
157+
158+
fn count_stroke_joins(&mut self, style: Join, width: f32, count: u32, scale: f32) {
159+
match style {
160+
Join::Bevel => self.lines.linetos += count,
161+
Join::Miter => self.lines.linetos += 2 * count,
162+
Join::Round => {
163+
self.lines.curves += count * estimate_arc_lines(width, scale);
164+
self.lines.curve_count += 1;
165+
}
166+
}
167+
}
168+
}
169+
170+
fn estimate_arc_lines(stroke_width: f32, scale: f32) -> u32 {
171+
// These constants need to be kept consistent with the definitions in `flatten_arc` in
172+
// flatten.wgsl.
173+
const MIN_THETA: f32 = 1e-4;
174+
const TOL: f32 = 0.1;
175+
let radius = TOL.max(scale * stroke_width * 0.5);
176+
let theta = (2. * (1. - TOL / radius).acos()).max(MIN_THETA);
177+
((std::f32::consts::FRAC_PI_2 / theta).ceil() as u32).max(1)
178+
}
179+
180+
#[derive(Clone, Default)]
181+
struct LineSoup {
182+
// Explicit lines (such as linetos and non-round stroke caps/joins) and Bezier curves
183+
// get tracked separately to ensure that explicit lines remain scale invariant.
184+
linetos: u32,
185+
curves: u32,
186+
187+
// Curve count is simply used to ensure a minimum number of lines get counted for each curve
188+
// at very small scales to reduce the chance of under-allocating.
189+
curve_count: u32,
190+
}
191+
192+
impl LineSoup {
193+
fn tally(&self, scale: f32) -> u32 {
194+
let curves = self
195+
.scaled_curve_line_count(scale)
196+
.max(5 * self.curve_count);
197+
198+
self.linetos + curves
199+
}
200+
201+
fn scaled_curve_line_count(&self, scale: f32) -> u32 {
202+
(self.curves as f32 * scale.sqrt()).ceil() as u32
203+
}
204+
205+
fn add(&mut self, other: &LineSoup, scale: f32) {
206+
self.linetos += other.linetos;
207+
self.curves += other.scaled_curve_line_count(scale);
208+
self.curve_count += other.curve_count;
209+
}
210+
}
211+
212+
// TODO: The 32-bit Vec2 definition from cpu_shaders/util.rs could come in handy here.
213+
fn transform(t: &Transform, v: Vec2) -> Vec2 {
214+
Vec2::new(
215+
t.matrix[0] as f64 * v.x + t.matrix[2] as f64 * v.y,
216+
t.matrix[1] as f64 * v.x + t.matrix[3] as f64 * v.y,
217+
)
218+
}
219+
220+
fn transform_scale(t: Option<&Transform>) -> f32 {
221+
match t {
222+
Some(t) => {
223+
let m = t.matrix;
224+
let v1x = m[0] + m[3];
225+
let v2x = m[0] - m[3];
226+
let v1y = m[1] - m[2];
227+
let v2y = m[1] + m[2];
228+
(v1x * v1x + v1y * v1y).sqrt() + (v2x * v2x + v2y * v2y).sqrt()
229+
}
230+
None => 1.,
231+
}
232+
}
233+
234+
/// Wang's Formula (as described in Pyramid Algorithms by Ron Goldman, 2003, Chapter 5, Section
235+
/// 5.6.3 on Bezier Approximation) is a fast method for computing a lower bound on the number of
236+
/// recursive subdivisions required to approximate a Bezier curve within a certain tolerance. The
237+
/// formula for a Bezier curve of degree `n`, control points p[0]...p[n], and number of levels of
238+
/// subdivision `l`, and flattening tolerance `tol` is defined as follows:
239+
///
240+
/// ```ignore
241+
/// m = max([length(p[k+2] - 2 * p[k+1] + p[k]) for (0 <= k <= n-2)])
242+
/// l >= log_4((n * (n - 1) * m) / (8 * tol))
243+
/// ```
244+
///
245+
/// For recursive subdivisions that split a curve into 2 segments at each level, the minimum number
246+
/// of segments is given by 2^l. From the formula above it follows that:
247+
///
248+
/// ```ignore
249+
/// segments >= 2^l >= 2^log_4(x) (1)
250+
/// segments^2 >= 2^(2*log_4(x)) >= 4^log_4(x) (2)
251+
/// segments^2 >= x
252+
/// segments >= sqrt((n * (n - 1) * m) / (8 * tol)) (3)
253+
/// ```
254+
///
255+
/// Wang's formula computes an error bound on recursive subdivision based on the second derivative
256+
/// which tends to result in a suboptimal estimate when the curvature within the curve has a lot of
257+
/// variation. This is expected to frequently overshoot the flattening formula used in vello, which
258+
/// is closer to optimal (vello uses a method based on a numerical approximation of the integral
259+
/// over the continuous change in the number of flattened segments, with an error expressed in terms
260+
/// of curvature and infinitesimal arclength).
261+
mod wang {
262+
use super::*;
263+
264+
// The curve degree term sqrt(n * (n - 1) / 8) specialized for cubics:
265+
//
266+
// sqrt(3 * (3 - 1) / 8)
267+
//
268+
const SQRT_OF_DEGREE_TERM_CUBIC: f64 = 0.86602540378;
269+
270+
// The curve degree term sqrt(n * (n - 1) / 8) specialized for quadratics:
271+
//
272+
// sqrt(2 * (2 - 1) / 8)
273+
//
274+
const SQRT_OF_DEGREE_TERM_QUAD: f64 = 0.5;
275+
276+
pub fn quadratic(rsqrt_of_tol: f64, p0: Vec2, p1: Vec2, p2: Vec2, t: &Transform) -> u32 {
277+
let v = -2. * p1 + p0 + p2;
278+
let v = transform(t, v); // transform is distributive
279+
let m = v.length();
280+
(SQRT_OF_DEGREE_TERM_QUAD * m.sqrt() * rsqrt_of_tol).ceil() as u32
281+
}
282+
283+
pub fn cubic(rsqrt_of_tol: f64, p0: Vec2, p1: Vec2, p2: Vec2, p3: Vec2, t: &Transform) -> u32 {
284+
let v1 = -2. * p1 + p0 + p2;
285+
let v2 = -2. * p2 + p1 + p3;
286+
let v1 = transform(t, v1);
287+
let v2 = transform(t, v2);
288+
let m = v1.length().max(v2.length()) as f64;
289+
(SQRT_OF_DEGREE_TERM_CUBIC * m.sqrt() * rsqrt_of_tol).ceil() as u32
290+
}
291+
}

0 commit comments

Comments
 (0)