[Dev] Installing pre-commit hooks fails due to PIP_USER=1

[PNOGillespie]

Describe the bug
Installation of pre-commit for AiiDALab-QE repository fails when trying to install Hooks, giving the following traceback:

[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for https://github.com/sirosen/check-jsonschema.
[INFO] Initializing environment for https://github.com/astral-sh/ruff-pre-commit.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/home/jovyan/.cache/pre-commit/repoj5uon3ys/py_env-python3.9/bin/python', '-mpip', 'install', '.')
return code: 1
stdout: (none)
stderr:
    ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
Check the log at /home/jovyan/.cache/pre-commit/pre-commit.log

To Reproduce
Following official instructions for installation via Docker container (either docker run aiidalab/qe:latest or aiidalab-launch with/without aiidalab-qe):

1. Pull latest version of aiidalab-qe docker build
2. Install AiiDALab-QE App with dev tools: pip install -e .[dev]
3. Run pre-commit: pre-commit (or pre-commit install-hooks)

Version (if known)

• App version: 24.10.0a
• Operating system: Ubuntu 24.04
• Docker engine version details:
  • Version: 27.2.1
  • API version: 1.47
  • Go version: go1.22.7
  • Git commit: 9e34c9b
  • Built: Fri Sep 6 12:08:02 2024
  • OS/Arch: linux/amd64
  • Context: default

[PNOGillespie]

Hi @superstar54, @edan-bainglass. I unfortunately encountered this problem while trying to address the request in aiidalab/aiidalab-qe#802. I'm also getting a persisting warning during installation for aiidalab-qe:

DEPRECATION: aiidalab-eln 0.1.2 has a non-standard dependency specifier aiida-core<3",>=1.1.0. pip 24.1 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of aiidalab-eln or contact the author to suggest that they release a version with a conforming dependency specifiers.

Not sure if it's relevant to this issue, but wanted to mention it anyway.

[edan-bainglass]

Hi @PNOGillespie. Regarding the deprecation, this was mentioned before. I believe @danielhollas was considering removing the dependency, though I think the discussion was pending a word from @yakutovicha.

Not sure about the other part. @superstar54?

[danielhollas]

The deprecation was solved some time ago and will be fixed in AWB=2.3.0.

Regarding the pre-commit fail, that is an unfortunate interaction with a change I did in Docker stack where we set the PIP_USER=1 envvar to instruct pip to default to --user install. Without this, pip would by default install into the conda environment at /opt/conda.

I didn't realise that this would impact pre-commit hooks. I think ultimately this is something that should be fixed in pre-commit. I'll try to open an issue. In the meantime, you can run export PIP_USER=0 and that should fix it.

[PNOGillespie]

Thanks for the reply @danielhollas, indeed it did fix the issue.

@superstar54, do you want to close this issue since @danielhollas is going to open a new one?

[danielhollas]

I am going to move this issue to aiidalab-docker-stack repo since that's where it originates

[danielhollas]

There is a (closed) issue about this on pre-commit repo, I commented on it, let's see

pre-commit/pre-commit#1151

[danielhollas]

pre-commit will not do anything so we'll need to fix this on our end, but I don't know how yet.

I thought that using an EXTERNALLY-MANAGED file in /opt/conda/lib/python3.9 would do the trick, as per the specification, but unfortunately that disallows user installs as well.

For my own posterity here's an unhelpful chat with Claude