Skip to content

Conversation

@alecthomas
Copy link
Collaborator

@alecthomas alecthomas commented Jan 16, 2026

Interestingly this doesn't seem to result in significant gains. It's also quite costly in terms of storage, so it's disabled by default. That said, it likely does reduce CPU utilisation on the cache server, as it can just stream the bundle back and not have to compute it on the fly.

I was planning to automatically inject bundle-uri commands in the prototocol response, but I think given the minimal improvements I'll defer that.

Completely uncached clone:

~/dev/cachew $ rm -rf linux-source ; time git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source
Cloning into 'linux-source'...
remote: Enumerating objects: 11276289, done.
remote: Counting objects: 100% (98/98), done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 11276289 (delta 68), reused 49 (delta 42), pack-reused 11276191 (from 1)
Receiving objects: 100% (11276289/11276289), 5.65 GiB | 7.55 MiB/s, done.
Resolving deltas: 100% (9153747/9153747), done.
Updating files: 100% (92194/92194), done.
git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source  486.00s user 112.07s system 67% cpu 14:50.60 total

Using the mirror on the cache server:

~/dev/cachew $ rm -rf linux-source ; time git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source
Cloning into 'linux-source'...
remote: Enumerating objects: 11084152, done.
remote: Counting objects: 100% (11084152/11084152), done.
remote: Compressing objects: 100% (2034375/2034375), done.
remote: Total 11084152 (delta 8997999), reused 11084152 (delta 8997999), pack-reused 0 (from 0)
Receiving objects: 100% (11084152/11084152), 5.58 GiB | 101.10 MiB/s, done.
Resolving deltas: 100% (8997999/8997999), done.
Updating files: 100% (92194/92194), done.
git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source  410.10s user 86.08s system 176% cpu 4:40.95 total

Using the mirror on the cache server AND the bundle:

~/dev/cachew $ rm -rf linux-source ; time git clone --bundle-uri http://127.0.0.1:8080/git/github.com/torvalds/linux.git/bundle http://127.0.0.1:8080/git/github.com/torvalds/linux.git linux-source
Cloning into 'linux-source'...
remote: Enumerating objects: 908, done.
remote: Counting objects: 100% (908/908), done.
remote: Compressing objects: 100% (908/908), done.
remote: Total 908 (delta 0), reused 908 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (908/908), 414.26 KiB | 17.26 MiB/s, done.
Checking connectivity: 908, done.
Updating files: 100% (92194/92194), done.
git clone --bundle-uri   linux-source  358.40s user 81.89s system 197% cpu 3:42.47 total

@alecthomas alecthomas requested a review from js-murph as a code owner January 16, 2026 10:55
Interestingly this doesn't seem to result in significant gains. It's
also quite costly in terms of storage, so it's disabled by default. That
said, it likely does reduce CPU utilisation on the cache server, as it
can just stream the bundle back and not have to compute it on the fly.

I was planning to automatically inject bundle-uri commands in the
prototocol response, but I think given the minimal improvements I'll
defer that.

```
~/dev/cachew $ rm -rf linux-source ; time git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source
Cloning into 'linux-source'...
remote: Enumerating objects: 11276289, done.
remote: Counting objects: 100% (98/98), done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 11276289 (delta 68), reused 49 (delta 42), pack-reused 11276191 (from 1)
Receiving objects: 100% (11276289/11276289), 5.65 GiB | 7.55 MiB/s, done.
Resolving deltas: 100% (9153747/9153747), done.
Updating files: 100% (92194/92194), done.
git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source  486.00s user 112.07s system 67% cpu 14:50.60 total

~/dev/cachew $ rm -rf linux-source ; time git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source
Cloning into 'linux-source'...
remote: Enumerating objects: 11084152, done.
remote: Counting objects: 100% (11084152/11084152), done.
remote: Compressing objects: 100% (2034375/2034375), done.
remote: Total 11084152 (delta 8997999), reused 11084152 (delta 8997999), pack-reused 0 (from 0)
Receiving objects: 100% (11084152/11084152), 5.58 GiB | 101.10 MiB/s, done.
Resolving deltas: 100% (8997999/8997999), done.
Updating files: 100% (92194/92194), done.
git clone http://127.0.0.1:8080/git/github.com/torvalds/linux linux-source  410.10s user 86.08s system 176% cpu 4:40.95 total

~/dev/cachew $ rm -rf linux-source ; time git clone --bundle-uri http://127.0.0.1:8080/git/github.com/torvalds/linux.git/bundle http://127.0.0.1:8080/git/github.com/torvalds/linux.git linux-source
Cloning into 'linux-source'...
remote: Enumerating objects: 908, done.
remote: Counting objects: 100% (908/908), done.
remote: Compressing objects: 100% (908/908), done.
remote: Total 908 (delta 0), reused 908 (delta 0), pack-reused 0 (from 0)
Receiving objects: 100% (908/908), 414.26 KiB | 17.26 MiB/s, done.
Checking connectivity: 908, done.
Updating files: 100% (92194/92194), done.
git clone --bundle-uri   linux-source  358.40s user 81.89s system 197% cpu 3:42.47 total
```
@alecthomas alecthomas changed the title feat: create bundle snapshots periodically feat: create git bundle snapshots periodically Jan 16, 2026
@alecthomas alecthomas merged commit 867d286 into main Jan 16, 2026
6 checks passed
@alecthomas alecthomas deleted the aat/create-bundles branch January 16, 2026 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants