feat: blobless clone by Owen-Liuyuxuan · Pull Request #92 · ros-infrastructure/vcs2l

Owen-Liuyuxuan · 2025-12-17T03:50:00Z

Basic Info

Info	Please fill out this column
Ticket(s) this addresses	resolves #91;
Primary OS tested on	Ubuntu
Is this a breaking change?	No
Does this PR contain AI generated software?	No;

Description of contribution in a few bullet points

For vcs import, add an optional argument blobless-clone
In git import:
- If there is no existing repository, it will perform a blobless clone, then require a follow-up checkout.
- If there is an existing repository, it will perform fetch with the blobless option.

Description of how this change was tested

run precommit before committing.
existing test passed.

Based on the current autoware repos, I run the following commands:

The resulting directory will have a size difference like this, but all required contents exist:

Signed-off-by: YuxuanLiuTier4Desktop <619684051@qq.com>

leander-dsouza

This is amazing :)
Thank you for raising the issue and proposing a valid solution 🎉

I would suggest adding a couple of tests for robustness, namely:

test_import_blobless
test_import_blobless_shallow_mutually_exclusive

PS. Since the repository uses local paths for tests, you would get the following warning while running --blobless-clone:

warning: filtering not recognized by server, ignoring

Feel free to include this warning as well in the expected file - import_blobless.txt.

leander-dsouza · 2026-02-20T18:20:01Z

vcs2l/commands/import_.py

The blobless_clone is an explicit parameter; hence, it needs to be exposed in the constructor.

Suggested change

def __init__(self, args, url, version=None, recursive=False, shallow=False, blobless_clone=False):

4ea438f Explicitly construction fixed here.

leander-dsouza · 2026-02-20T18:20:46Z

vcs2l/commands/import_.py

        self.skip_existing = args.skip_existing
        self.recursive = recursive
        self.shallow = shallow
+        self.blobless_clone = args.blobless_clone


This is a follow-up from the suggestion that requires blobless_clone to be explicit:

Suggested change

self.blobless_clone = args.blobless_clone

self.blobless_clone = blobless_clone

4ea438f Explicitly construction fixed here.

leander-dsouza · 2026-02-20T18:22:41Z

vcs2l/commands/import_.py

            repo['url'],
            str(repo['version']) if 'version' in repo else None,
            recursive=args.recursive,
            shallow=args.shallow,


The blobless_clone needs to be added in the ImportCommand.

Suggested change

shallow=args.shallow,

blobless_clone=args.blobless_clone,

4ea438f Explicitly construction fixed here.

leander-dsouza · 2026-02-20T18:39:15Z

vcs2l/commands/import_.py

    except (RuntimeError, request.URLError) as e:
        print(ansi('redf') + str(e) + ansi('reset'), file=sys.stderr)
        return 1
    jobs = generate_jobs(repos, args)


The --shallow and --blobless-clone tags need to be mutually exclusive, as when combined, we would neither get file history nor file content:

--shallow --> git clone --depth 1: Downloads only 1 commit but fetches all file blobs at that commit. You get a complete working tree but no history.

--blobless-clone --> git clone --filter=blob:none: Downloads the full commit history but no file blobs upfront. Blobs are fetched on-demand during checkout.

Suggested change

if args.shallow and args.blobless_clone:

print(

ansi('redf')

+ "'--shallow' and '--blobless-clone' are mutually exclusive options"

+ ansi('reset'),

file=sys.stderr,

)

return 1

jobs = generate_jobs(repos, args)

Thanks a lot! Exclusive fixed here:9ae0833

leander-dsouza · 2026-02-20T18:49:29Z

vcs2l/clients/git.py

+                    cmd_clone += ['--filter=blob:none', '--no-checkout']
+                    checkout_version = command.version


The earlier method creates a detached HEAD:

git clone --filter=blob:none --no-checkout https://github.com/org/repo . git checkout heads/main --

In order to get a proper local branch, support with -b needs to be maintained:

Suggested change

cmd_clone += ['--filter=blob:none', '--no-checkout']

checkout_version = command.version

cmd_clone.append('--filter=blob:none')

if version_type == 'branch':

cmd_clone += ['-b', version_name]

checkout_version = None

elif command.version:

cmd_clone.append('--no-checkout')

checkout_version = command.version

else:

checkout_version = None

a15ab81
46bc9ce

Fix the support with -b

Signed-off-by: Yuxuan Liu <619684051@qq.com>

Owen-Liuyuxuan · 2026-02-24T04:44:27Z

@leander-dsouza Thanks for the thorough review. I added the fixes required.

For the test,

python3 -m unittest test.test_commands.TestCommands.test_import_blobless test.test_commands.TestCommands.test_import_blobless_shallow_mutually_exclusive Locally test passed.
I used AI to generate the test codes.
- The test content looks legit.
- Warning were originally fixed in the codes -- I prompted AI to transfer the warning texts into the "expected outputs" as required by the review.

feat: blobless clone

133aa29

Signed-off-by: YuxuanLiuTier4Desktop <619684051@qq.com>

Owen-Liuyuxuan requested review from claraberendsen and leander-dsouza as code owners December 17, 2025 03:50

leander-dsouza requested changes Feb 20, 2026

View reviewed changes

leander-dsouza added the enhancement New feature or request label Feb 21, 2026

Owen-Liuyuxuan added 5 commits February 24, 2026 13:00

feat: explicitly added blobless clone in constructor

4ea438f

Signed-off-by: Yuxuan Liu <619684051@qq.com>

feat: shallow & blobless clone mutually exclusive check

9ae0833

Signed-off-by: Yuxuan Liu <619684051@qq.com>

feat: handle branch and version correctly in blobless clone

a15ab81

Signed-off-by: Yuxuan Liu <619684051@qq.com>

feat: add test and correctly handle checked_version

46bc9ce

Signed-off-by: Yuxuan Liu <619684051@qq.com>

fix: fix not needed env

96f7433

Signed-off-by: Yuxuan Liu <619684051@qq.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: blobless clone#92

feat: blobless clone#92
Owen-Liuyuxuan wants to merge 6 commits intoros-infrastructure:mainfrom
Owen-Liuyuxuan:feat/blobless_clone

Owen-Liuyuxuan commented Dec 17, 2025 •

edited

Loading

Uh oh!

leander-dsouza left a comment

Uh oh!

leander-dsouza Feb 20, 2026

Uh oh!

Owen-Liuyuxuan Feb 24, 2026

Uh oh!

leander-dsouza Feb 20, 2026

Uh oh!

Owen-Liuyuxuan Feb 24, 2026

Uh oh!

leander-dsouza Feb 20, 2026

Uh oh!

Owen-Liuyuxuan Feb 24, 2026

Uh oh!

leander-dsouza Feb 20, 2026

Uh oh!

Owen-Liuyuxuan Feb 24, 2026

Uh oh!

leander-dsouza Feb 20, 2026

Uh oh!

Owen-Liuyuxuan Feb 24, 2026

Uh oh!

Owen-Liuyuxuan commented Feb 24, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


	def __init__(self, args, url, version=None, recursive=False, shallow=False, blobless_clone=False):

	self.blobless_clone = args.blobless_clone
	self.blobless_clone = blobless_clone

		cmd_clone += ['--filter=blob:none', '--no-checkout']
		checkout_version = command.version

-                    cmd_clone += ['--filter=blob:none', '--no-checkout']
-                    checkout_version = command.version
+                    cmd_clone.append('--filter=blob:none')
+                    if version_type == 'branch':
+                        cmd_clone += ['-b', version_name]
+                        checkout_version = None
+                    elif command.version:
+                        cmd_clone.append('--no-checkout')
+                        checkout_version = command.version
+                    else:
+                        checkout_version = None

Conversation

Owen-Liuyuxuan commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Basic Info

Description of contribution in a few bullet points

Description of how this change was tested

Uh oh!

leander-dsouza left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Owen-Liuyuxuan commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Owen-Liuyuxuan commented Dec 17, 2025 •

edited

Loading

Owen-Liuyuxuan commented Feb 24, 2026 •

edited

Loading