PyPI Insiders

Self-hosted PyPI server with automatic updates for Insiders versions of projects.

Note

This project will soon be integrated into https://github.com/pawamoy/insiders-project.

Motivation

Some open source projects follow the sponsorware release strategy, which means that new features are first exclusively released to sponsors as part of an "Insiders" version of the project. This Insiders version is usually a private fork of the public project.

To use the Insiders projects as dependencies, sponsors have two options:

1. specify the dependency as a Git URL (HTTPS or SSH), or as a direct HTTPS URL to a build artifact
2. build and store the artifact in a self-hosted PyPI-like index

The first option is problematic when sponsors' projects are also open source, because most of their contributors will probably not have access to the Insiders version. It means they won't be able to resolve the dependency, even less install it locally. As a result, maintainers must specify the public version of the project as a dependency, and override it with the Insiders version in Continuous Integration / Deployment.

In contrast, the second option allows maintainers to specify the dependency normally, i.e. using the same name/identifier as the public version. Maintainers/contributors with access to the Insiders version will resolve and get the Insiders version, while maintainers/contributors without access to the Insiders version will simply get the public one.

However, self-hosting a PyPI-like index, and building artifacts for each new Insiders version is not a trivial, straight-forward task: companies and organizations might already have such a setup (with an Artifactory server, a Google Cloud registry, etc.), but individual contributors often won't, and automatically updating repositories, building artifacts and uploading them requires custom scripts.

In both cases (company setup or individual contributor) PyPI Insiders comes to the rescue, and manages repository/package updates for you. It comes bundled with a PyPI-like index that you can serve locally, and it watches Insiders repositories, building and uploading distributions to your local index (or any other online index) for each new Insiders version getting published.

See below how to install and use PyPI Insiders!

Installation

pip install pypi-insiders

With uv:

uv tool install pypi-insiders

Usage

Run the server with:

pypi-insiders server start

The local PyPI server should be running:

pypi-insiders server status

Now, if you wish, you can configure your tools to use your local index by default:

export PIP_INDEX_URL=http://localhost:31411/simple/
export PDM_PYPI_URL=http://localhost:31411/simple/
export UV_INDEX_URL=http://localhost:31411/simple/

Your local index will give precedence to its own packages, and redirect to PyPI.org if it doesn't know the specified packages. It means that Insiders versions will always take precedence over public versions, even if the latter are higher (more recent).

Configuring your tools with environment variables makes it easy to temporarily "deactivate" your local index:

# This will install directly from PyPI.org.
env -u PIP_INDEX_URL pip install something
env -u PDM_PYPI_URL pdm sync
env -u UV_INDEX_URL uv sync

You can declare a shell alias to make things even simpler:

alias no-insiders='env -u PIP_INDEX_URL -u PDM_PYPI_URL -u UV_INDEX_URL'
no-insiders uv sync

Configure the repositories to watch:

pypi-insiders repos add pawamoy-insiders/devboard:devboard

The format is NAMESPACE/PROJECT:DISTRIBUTION_NAME. Only GitHub projects are supported for now.

List watched repositories:

pypi-insiders repos list

Remove watched repositories:

pypi-insiders repos remove pawamoy-insiders/devboard

Start/stop the local PyPI index, get the server status:

pypi-insiders server start
pypi-insiders server status
pypi-insiders server stop

Update all packages from watched repositories:

pypi-insiders update

Update a specific package:

pypi-insiders update pawamoy-insiders/devboard

Start/stop the watcher, get the watcher status:

pypi-insiders watcher start
pypi-insiders watcher status
pypi-insiders watcher stop

Show logs of the server/watcher:

pypi-insiders server logs
pypi-insiders watcher logs