gitlist-sync

🔄 Setup and sync mirrors of GitHub repositories to local machine

This repo contains the scripts and webhook handling server for setting up and syncing of the mirrors of my public GitHub repositories (owned by me, no forks). Further, I use GitList to host and publicly avail them.

My mirrors: https://git.mihir.ch.

Configuration

config.json

1. Create config.json file from config.example.json: cp config.example.json config.json

• repositoriesPath is the local path where your git repos are going to be cloned and stored.
• webhookBase is the base url to which the webhook will be attached.
• githubRepoListParams won't really need much modification, unless you really want to. Read this.

.env

1. Create .env file from .env.example: cp .env.example .env
2. Add the required variables. Note: the GitHub token will require the repo and admin:repo_hook scopes.
3. Create a strong secret for the WebHook.

Instructions

This project uses Node. Install dependencies: npm install.

Setup

Run: node scripts/setup

Setting up is handled by the setup.js file:

The script:

• Clones all of my public repositories;
• Adds, for each cloned repo, the GitHub desciption to the .git/description file;
• Creates, for each cloned repo, a GitHub webhook for the push event that triggers a git pull on the respective repo whenever a new commit is pushed to the remote GitHub repo.

Webhook Server

Run: node webhook-server

This server listens for requests on the payload URLs (specified by config.webhookBase + /push) and triggers a git pull on the repo that got pushed to, retrieved from the webhook payload. Read about GitHub webhooks here.

The webhook is protected with secret, specified when programmatically creating the webhook in the Setup step. Signature verification ensures request integrity.

Suggestion: use pm2 or equivalent to daemonize the server.

Sync

Run: node scripts/sync

Syncing is handled by the sync.js.

It checks the updated repo list against the repositories.json file (auto-generated) and performs clones for new repositories and deletions for deleted repos. Webhooks are not deleted.

Suggestion: this is a one-time script. I suggest setting up a cron for this script. I have it as 0 0 * * * (daily).

License

MIT