Suggestion: shared git store for nixpkgs on community builders

[nbraud]

Hi!

A minor-but-recurring annoyance I encountered on the community builders, is the time it takes to update nixpkgs, especially after the system was reset (as the repo has to be cloned from scratch)

I'd like to suggest keeping a bare repo, under e.g. /var/git/nixpkgs:

• containing a subset of nixpkgs' refs, like master, release-* and staging{,-*} ;
• periodically updated, at whichever frequency works out best ;
• readable by users.

Then, we can significantly speed up cloning & updating nixpkgs, as well as saving significants amounts of disk space, by referring git to that local “mirror” :

$ git clone --reference /var/git/nixpkgs https://github.com/NixOS/nixpkgs.git

The repository behaves like a completely-normal git clone, but will reuse objects from /var/git/nixpkgs even on future updates (as .git/objects/info/alternates references it)

Quickly testing on the Darwin box, assuming a (bare) repo mirroring nixpkgs is available locally, getting a fresh clone takes under 30 s and 350 MiB of disk space, saving ~5 GiB per clone of nixpkgs on the system; furthermore, the benefits will only grow with nixpkgs' history.

Note: it's important that objects present in /var/git/nixpkgs never disappear; that should be the case simply because the nixpkgs refs that it mirrors are never force-pushed to, but it might be worth configuring the bare repo not to GC unreachable objects.

[nbraud]

If that would be a welcome addition, I could try writing a PR for it, though I'd need to be pointed at the documentation for available options on Darwin.

PS: nevermind, I found nix-darwin, though I might still mess up since I can't easily test things.

[zowoq]

I've opened a draft PR for this I still need to handle fetching/updating the git refs. #1455