Data size SKU for billing

[gefjon]

Description of Changes

This PR adds the ability to compute and report the number of rows in memory, and the number of bytes used by those rows.

Part of https://github.com/clockworklabs/SpacetimeDBPrivate/issues/1229 .

Currently, reporting is accomplished by a new group of metrics, all of which are prefixed with spacetime_data_size.

Specifically, for each database, we report:

• blob_store_num_blobs, the number of bytes used by large blobs in the BlobStore.
  • We just report the actual size in bytes of the blobs.
    • Strings' sizes are len(str).
    • Other blobs' sizes are the BSATN size.
  • Duplicated blobs are counted multiple times, despite their being deduplicated by hash in the storage layer. I believe this to be more predictable to clients, since it preserves the property that inserting a row which contains a large blob causes a string increase in the metric, and deleting such a row causes a strict decrease.
• blob_store_bytes_used_by_blobs, the number of blobs in the BlobStore.
  • The flat overhead per blob is likely negligible compared to the data cost of those blobs, given that they start at 1 KiB and the overhead is likely on the order of 64 bytes, but we measure and report it for completeness.
  • As with blob-bytes, this counts duplicated blobs multiple times.

For each table in each database, we report:

• table_num_rows, the number of rows in the table.
• Flat overhead here may be more meaningful, since many tables have quite small row sizes.
• table_bytes_used_by_rows, the number of bytes in Pages used by rows in the table.
  • This is the sum of the fixed-len size and the granules allocated to store var-len portions for the rows.
  • Granules are always 32 bytes long; fixed-len size is a per-table constant.
  • Overhead due to padding, "pointer-like" contents, &c is included.
  • Size of large blobs is not included here (see above).
  • Free space in Pages is also not included here.
• table_num_rows_in_indexes, the number of rows in indexes in the table.
  • We could hypothetically partition this by index. That wouldn't be very interesting, though, because it's just table_size * num_indices.
  • Again, flat overhead potentially matters here.
• table_bytes_used_by_index_keys, the bytes used to store keys in indexes in the table.
  • We could hypothetically partition this by index. Unlike above, that would potentially be interesting.
  • See doc comments on the new KeySize trait for a precise definition of this metric.

In this PR, the new metrics are reported when committing a mutable TX, as that's when their values change. However, it's not necessary to report them this often; unlike our existing metrics they are not incremental. (Or the incremental maintenance is confined to the table crate, and not visible to the core crate where they are read and reported.) It would be reasonable to report them every N transactions for some choice of N, or every t seconds for some choice of t, or in response to an external request, or in any number of ways.

API and ABI breaking changes

N/a, unless adding metrics breaks Prometheus in some way I don't understand.

Expected complexity level and risk

3: it would be unfortunate if we misreported these, since we intend to use them for billing, and the computations for some of the new metrics are non-trivial. It's also possible (I haven't checked) that reporting the benchmarks will have meaningful overhead, causing a performance regression. That said, these changes are very unlikely to break any existing functionality.

Testing

• Wrote proptests that the low-level measurements of number and bytes used by rows and indexes is as expected, compared to an obviously correct (IMO) naiive implementation.
• I am not sure how to test these manually, as I don't know how to run locally while collecting Prometheus metrics.

[Centril]

Still a draft, but the overall strategy here makes sense. :)

[jsdt]

This looks good overall. Adding testing is important if we are going to use these for billing.

Is it possible to write a function that computes the size of a row s.t. we can assert that bytes_used_by_rows() is equal to the sum of the size of each row? If so, that would give us a good path toward being able to write tests.

[jsdt]

If this causes a performance issue because of time spent in with_label_values, we could store these metric handles in the Table and/or ExecutionContext, so reporting values would just be an atomic number operation.

[jsdt]

In the code immediately above this, we are also updating a bunch of stats (counters and table size gauges). We should probably unify those at some point.

[gefjon]

@jsdt I have written some unit proptests in table.rs, row_size_reporting_matches_slow_implementations, index_size_reporting_matches_slow_implementations_single_column and ibid _two_column. I have left comments that I am not sure there is a meaningful way to proptest the blob store reporting, since it's already as stupid and obvious an impl as possible. Is there anything else you'd like to see?

[jsdt]

These tests look good, thanks for adding them. For the blob store, do you think recomputing it every time we report is going to be a performance issue? If so, it might be worth keeping track of it as modifications happen.

[gefjon]

These tests look good, thanks for adding them. For the blob store, do you think recomputing it every time we report is going to be a performance issue? If so, it might be worth keeping track of it as modifications happen.

I would assume that the majority of modules have very small blob stores. If this turns out not to be the case we could easily memoize it in the same way as this PR is doing for other measures.