Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix poor performance of to_tracking_manifest #1047

Merged
merged 1 commit into from
Jun 14, 2024
Merged

Conversation

jrray
Copy link
Collaborator

@jrray jrray commented Jun 13, 2024

This method had something like O(n^3) complexity because it would for
every tree in a manifest iterate over every tree in a manifest and
calculate the digest for every tree in the manifest (modulo short
circuiting).

Manifest::get_tree is the culprit and an alternative technique of
caching the mapping of digest to tree was added. Both uses of this
method have been updated but this method was left in place with a new
warning. It would be cheaper to call get_tree in a contrived case
where the cache is only accessed once and not reused for multiple
lookups.

Fixes #1043.

@jrray jrray self-assigned this Jun 13, 2024
@jrray jrray force-pushed the to-tracking-manifest-perf branch from d195019 to e6fb537 Compare June 13, 2024 01:41
@jrray jrray mentioned this pull request Jun 13, 2024
@jrray jrray force-pushed the to-tracking-manifest-perf branch from e6fb537 to ff99801 Compare June 13, 2024 01:55
This method had something like O(n^3) complexity because it would for
every tree in a manifest iterate over every tree in a manifest and
calculate the digest for every tree in the manifest (modulo short
circuiting).

`Manifest::get_tree` is the culprit and an alternative technique of
caching the mapping of digest to tree was added. Both uses of this
method have been updated but this method was left in place with a new
warning. It would be cheaper to call `get_tree` in a contrived case
where the cache is only accessed once and not reused for multiple
lookups.

Fixes #1043.

Signed-off-by: J Robert Ray <jrray@imageworks.com>
@jrray jrray force-pushed the to-tracking-manifest-perf branch from ff99801 to 4b80569 Compare June 13, 2024 01:57
@jrray jrray merged commit f6c1c8f into main Jun 14, 2024
7 checks passed
@jrray jrray deleted the to-tracking-manifest-perf branch June 14, 2024 16:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

flatbuffers unusably slow
2 participants