process: sync commits from Aspect-internal silo #781
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
- Covered by existing test cases GitOrigin-RevId: b77e25c5dce88015b2d7a80d78ba14203590153b
--- ### Changes are visible to end-users: no ### Test plan - Manual testing; please provide instructions so we can reproduce: bazel test //workflows/bessie/test:e2e --test_output=streamed --test_filter="correlates failing actions with target completions" GitOrigin-RevId: fd87dbf9b0cdcc7a6620f47798ff6ce1bc410e72
`sync.Map` seems optimal for this exact use case where caching has no evictions and only grows. > The Map type is optimized for two common use cases: (1) when the entry for a given key is only ever written once but read many times, as in caches that only grow, or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys. In these two cases, use of a Map may significantly reduce lock contention compared to a Go map paired with a separate [Mutex](https://pkg.go.dev/sync#Mutex) or [RWMutex](https://pkg.go.dev/sync#RWMutex). --- ### Changes are visible to end-users: no ### Test plan - Covered by existing test cases GitOrigin-RevId: 79fb097ab5bee25dcd6c7b9e1f760dc8dcc92753
…#7296) This avoids putting the entire lockfile into memory all at once just to iterate over it once and throw it away. --- ### Changes are visible to end-users: yes - Searched for relevant documentation and updated as needed: yes - Breaking change (forces users to change their own code or config): no - Suggested release notes appear below: yes Memory improvements to js `aspect configure` ### Test plan - Covered by existing test cases GitOrigin-RevId: ac5ef37ffcdba9db4108632d62fdfc9f4de8b981
Caching the parsing + compiling of jq queries. --- - Covered by existing test cases GitOrigin-RevId: c49200173440e195c41fc58894ddcfa9539127e9
…7326) These simple string operations are sometimes noticeable in memory profiles, at least in the utils invoked multiple times per file in repos of 50k+ files. --- ### Changes are visible to end-users: no ### Test plan - Covered by existing test cases GitOrigin-RevId: ff0100238cbe0b77d54c629456ffe2678ccf8342
…7335) Simply invoking `node.Type()` 3x for every root node for 45k .ts files was noticeable when memory profiling. --- ### Changes are visible to end-users: no ### Test plan - Covered by existing test cases GitOrigin-RevId: 89e73b1040c33a451c2cb3778c236b454b102a39
The patch changes are primarily due to bazel-contrib/bazel-gazelle@ed19735 --- ### Changes are visible to end-users: no ### Test plan - Covered by existing test cases GitOrigin-RevId: bed62a066c8ab87b77cabee01eadd2ffaf718d9b
…(#7357) EXPERIMENTAL: An initial internal API for caching data to disk between aspect configure runs. This must be manually enabled with the `ASPECT_CONFIGURE_CACHE` env var which points to the cache file. Initially this is being used by the js and starzelle languages to cache source code queries, primarily tree-sitter parse+query but also things like `jq` or regex queries are covered in the starzelle case. On a large customer repo this reduces the total CPU time from ~55s to 12s, although the wall time the user sees is more around 16s to 9s (= 40% faster, roughly aligning with my profiling showing). Examples, pre-cache: ``` 56.08s user 14.16s system 437% cpu 16.056 total 56.73s user 16.21s system 410% cpu 17.765 total ``` Post-cache: ``` 11.69s user 15.04s system 295% cpu 9.049 total 12.12s user 16.74s system 305% cpu 9.459 total 12.05s user 8.44s system 202% cpu 10.133 total ``` --- ### Changes are visible to end-users: yes - Searched for relevant documentation and updated as needed: yes - Breaking change (forces users to change their own code or config): no - Suggested release notes appear below: yes Add experimental caching of source code analysis for `aspect configure` via `ASPECT_CONFIGURE_CACHE` environment variable pointing to a cache file path, normally in a temp or ignored file location. ### Test plan - Covered by existing test cases - Manual testing; please provide instructions so we can reproduce: run `aspect configure` on a large repo with the env var GitOrigin-RevId: 399d18a70277a58ed56288ceeb6e2f1ff025a9e6
jbedard
force-pushed
the
788EA150BB0FAAB4A3FF81F26DCBE189
branch
from
bc6caa5 to
31aef06
Compare
gregmagolan
approved these changes
Nov 30, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Commit range https://github.com/aspect-build/silo/compare/08aa6c34918dd9858a4124e77274fd816745c0cd..b77e25c5dce88015b2d7a80d78ba14203590153b