Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add input parameter filter for git commands #3595

Merged
merged 3 commits into from
Jan 17, 2025
Merged

Add input parameter filter for git commands #3595

merged 3 commits into from
Jan 17, 2025

Conversation

emcfarlane
Copy link
Contributor

This adds a new input parameter filter to assist in working with large git projects. The param filter sets the git filter-spec argument when fetching repositories. Only one argument can be provided, the combined form can be used to set multiple filters. When using filter combined with a subdir the checkout will now also be sparse to further reduce the number of workspace files.

The documentation for filter arguments can be found in the git rev-list command: https://git-scm.com/docs/git-rev-list#Documentation/git-rev-list.txt---filterltfilter-specgt
See this blog post on the different filter checkouts that may be used: https://github.blog/open-source/git/get-up-to-speed-with-partial-clone-and-shallow-clone/

As an example using the envoy project as a larger repository, it can be built using the new filter argument:

 buf build https://github.com/envoyproxy/envoy.git#subdir=api,branch=main,filter=tree:0

Which reduces the fetch of the git repository from 10.6s to 2.6s.

This adds a new input parameter filter to assist in working with large
git projects. The param filter sets the git filter-spec argument when
fetching repositories. Only one argument can be provided, the combined
form can be used to set multiple filters. When using filter combined
with a subdir the checkout will now also be sparse to further reduce the
number of workspace files.
Copy link
Contributor

github-actions bot commented Jan 16, 2025

The latest Buf updates on your PR. Results from workflow Buf CI / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedJan 16, 2025, 6:14 PM

@emcfarlane emcfarlane marked this pull request as ready for review January 16, 2025 18:18
@doriable doriable self-requested a review January 16, 2025 20:36
Copy link
Member

@bufdev bufdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure you update docs too

@emcfarlane emcfarlane merged commit 73a8c77 into main Jan 17, 2025
10 checks passed
@emcfarlane emcfarlane deleted the ed/gitFilter branch January 17, 2025 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants