-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Remote Cache for golangci-lint #4986
Comments
Note this feature relies on golang/go#64876 |
Just to clarify why I put It requires a lot of time to ensure it is non-breaking and provides a reviewable PR. My priority is maintainability, so the new feature (if not added by the sync) will be evaluated after the sync because this a really really really long and complex job to just re-sync the code. The changes in the original code have never been documented, I already added a "summary" but this is not enough to work quickly on this. |
Did not realize you were refactoring or reimplementing the caching behaviour. Will wait for that and then we can see how to extend it for remote cache. |
@ryancurrah I updated the internals inside #5098. As a side effect, the programmatic cache is supported but I didn't test it. If you plan to create a cache implementation, I'm interested in playing with it or contributing. |
I didn't saw before but there is a test implementation: https://github.com/bradfitz/go-tool-cache/ $ go install github.com/bradfitz/go-tool-cache/cmd/go-cacher@latest
go: downloading github.com/bradfitz/go-tool-cache v0.0.0-20230425225207-ef6c7b1b26e9
$ GOLANGCI_LINT_CACHEPROG="$(which go-cacher) --verbose" ./golangci-lint run
2024/11/01 19:11:42 Defaulting to cache dir /home/ldez/.cache/go-cacher ...
2024/11/01 19:12:03 cacher: closing; 8009 gets (0 hits, 8009 misses, 0 errors); 9233 puts (0 errors)
$ GOLANGCI_LINT_CACHEPROG="$(which go-cacher) --verbose" ./golangci-lint run
2024/11/01 19:12:05 Defaulting to cache dir /home/ldez/.cache/go-cacher ...
2024/11/01 19:12:06 cacher: closing; 153 gets (153 hits, 0 misses, 0 errors); 0 puts (0 errors) And it works as expected, it's funny. |
Crazy good sir. Sorry I didn't get to review it. I will definitely try it out though. We have a large staging environment for testing all our CI tools. I can test this release on 30k builds in varying projects. |
Welcome
Your feature request related to a problem? Please describe
Yes, this feature request is related to the problem of long linting times, especially in large codebases or CI environments where there are multiple runners/agents.
Currently, golangci-lint lacks a mechanism to cache results remotely, which could help in reducing the overall time taken for linting when running it repeatedly over the same or similar code. This lack of caching increases build times and resource usage, especially when dealing with distributed and large build environments.
Describe the solution you'd like
I propose introducing a feature in golangci-lint to support remote caching by abstracting the cache mechanism. Similar to how Go has implemented the GOCACHEPROG environment variable, which allows a custom cache implementation via a child process that communicates using JSON over stdin/stdout, golangci-lint could introduce a GOLANGCICACHEPROG environment variable. This would allow users to specify a binary responsible for handling cache operations, thereby enabling remote caching similar to https://go-review.googlesource.com/c/go/+/486715.
Describe alternatives you've considered
Persisting cache on individual agents. In our environment, multiple agents are used, and they may not always have an up-to-date cache, leading to inconsistencies and longer build times.
Additional context
Willing to implement this myself if it is deemed an acceptable proposal. Remote caching has proven for us to reduce build times in Go projects. By using a similar approach in golangci-lint, users could experience similar reductions in linting times, especially in CI/CD environments or large projects when linting a lot of code. The ability to cache results remotely would lead to more efficient use of resources and faster feedback loops during builds.
Supporter
The text was updated successfully, but these errors were encountered: