Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add urllib3 v2 as network backend #292

Merged
merged 43 commits into from
Nov 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
0019125
requirements: add urllib3
R1kaB3rN Nov 29, 2024
dd24b55
pyproject: update dependencies
R1kaB3rN Nov 29, 2024
3e7ed8d
Makefile: add new build target for urllib3 v2
R1kaB3rN Nov 29, 2024
41a373c
__main__: add vendored path to PYTHONPATH
R1kaB3rN Nov 29, 2024
5298aba
umu_util: remove helper to create HTTPSConnection instances
R1kaB3rN Nov 29, 2024
bcb29d6
umu_util: add helper to compute digest and write file
R1kaB3rN Nov 29, 2024
30e5556
umu_run: refactor to use connection pools from urllib3 v2
R1kaB3rN Nov 29, 2024
b19d37b
umu_runtime: refactor to use urllib3 v2
R1kaB3rN Nov 29, 2024
ee22e58
umu_proton: refactor to use urllib3 v2
R1kaB3rN Nov 29, 2024
429e510
umu_test: add tests for _restore_umu
R1kaB3rN Nov 29, 2024
e346ab5
umu_test: add tests for setup_umu
R1kaB3rN Nov 29, 2024
436e691
umu_test: add tests for _restore_umu_platformid
R1kaB3rN Nov 29, 2024
022d385
umu_test: add tests for write_file_chunks
R1kaB3rN Nov 29, 2024
398b0c0
umu_test: update tests
R1kaB3rN Nov 29, 2024
8c13bdd
umu_test_plugins: update tests
R1kaB3rN Nov 29, 2024
b9fe434
workflows: setup venv
R1kaB3rN Nov 29, 2024
8926219
workflows: activate venv
R1kaB3rN Nov 29, 2024
4318b37
workflows: don't use fish shell
R1kaB3rN Nov 29, 2024
6838f02
workflows: don't append .sh to activate script
R1kaB3rN Nov 29, 2024
3f90154
workflows: setup venv for e2e
R1kaB3rN Nov 29, 2024
96980d1
workflows: fix venv in e2e
R1kaB3rN Nov 29, 2024
450b4bb
workflows: setup venv once
R1kaB3rN Nov 29, 2024
15b21ec
workflows: fix unit tests not running in venv
R1kaB3rN Nov 29, 2024
83b0e18
workflows: fix unit tests not running in venv
R1kaB3rN Nov 29, 2024
d6c367a
workflows: remove python 3.10 from matrix
R1kaB3rN Nov 29, 2024
b2a253d
workflows: fix mypy not running in venv
R1kaB3rN Nov 29, 2024
e75eb28
workflows: fix mypy not running in venv
R1kaB3rN Nov 29, 2024
b60384d
workflows: set python 3.11 for mypy
R1kaB3rN Nov 29, 2024
5d81fab
umu_runtime: add ignore for mypy
R1kaB3rN Nov 29, 2024
87a59aa
Makefile: fix deb build
R1kaB3rN Nov 29, 2024
04733bb
Makefile: fix format
R1kaB3rN Nov 29, 2024
0ab5838
deb: add python3-pip as build dependency
R1kaB3rN Nov 29, 2024
5ef4cb3
README: mention pip as build dependency
R1kaB3rN Nov 29, 2024
9c4515f
pyproject: add comment regarding urllib3 dep
R1kaB3rN Nov 29, 2024
45d68d7
pyproject: update comment for truststore
R1kaB3rN Nov 29, 2024
3cc3e0e
umu_runtime: update log statement
R1kaB3rN Nov 29, 2024
8b9be9a
umu_runtime: remove cached file on digest mismatch
R1kaB3rN Nov 29, 2024
790a10f
umu_runtime: update error message on digest mismatch
R1kaB3rN Nov 29, 2024
bfd6f71
umu_run: handle the digest mismatch when updating runtime
R1kaB3rN Nov 29, 2024
65048e1
umu_runtime: include the upstream link in the error message
R1kaB3rN Nov 29, 2024
aeea7b1
umu_proton: delete cached file on digest mismatch
R1kaB3rN Nov 29, 2024
34ac322
umu_runtime: extend archive file with its id to avoid corruption on r…
R1kaB3rN Nov 29, 2024
c4912ba
__main__: prefer list.insert when extending PYTHONPATH
R1kaB3rN Nov 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 42 additions & 36 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ name: e2e

on:
push:
branches: [ "main" ]
branches: ["main"]
pull_request:
branches: [ "main" ]
branches: ["main"]

permissions:
contents: read
Expand All @@ -14,37 +14,43 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install dependencies
run: |
sudo apt-get install meson scdoc python3-hatchling python3-build python3-installer python3-filelock shellcheck
python3 -m pip install --upgrade pip
pip install uv
- name: Initialize submodules
run: |
git submodule update --init --recursive
- name: Make user install
run: |
./configure.sh --user-install
make install
- name: Run shellcheck
run: |
shellcheck tests/*.sh
- name: Test steamrt install
run: |
sh tests/test_install.sh
rm -rf "$HOME/.local/share/umu" "$HOME/Games/umu" "$HOME/.local/share/Steam/compatibilitytools.d"
- name: Test steamrt update
run: |
sh tests/test_update.sh
rm -rf "$HOME/.local/share/umu" "$HOME/Games/umu" "$HOME/.local/share/Steam/compatibilitytools.d"
- name: Test winetricks
run: |
sh tests/test_winetricks.sh
rm -rf "$HOME/.local/share/umu" "$HOME/.local/share/Steam/compatibilitytools.d"
- name: Test configuration file
run: |
uv python install 3.11
uv run --python 3.11 -- sh tests/test_config.sh
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install dependencies
run: |
sudo apt-get install meson scdoc python3-hatchling python3-build python3-installer python3-filelock shellcheck
python3 -m pip install --upgrade pip
pip install uv
- name: Initialize submodules
run: |
git submodule update --init --recursive
- name: Make user install
run: |
./configure.sh --user-install
make install
- name: Run shellcheck
run: |
shellcheck tests/*.sh
- name: Setup venv
run: |
uv venv --python 3.11
- name: Test steamrt install
run: |
source .venv/bin/activate
sh tests/test_install.sh
rm -rf "$HOME/.local/share/umu" "$HOME/Games/umu" "$HOME/.local/share/Steam/compatibilitytools.d"
- name: Test steamrt update
run: |
source .venv/bin/activate
sh tests/test_update.sh
rm -rf "$HOME/.local/share/umu" "$HOME/Games/umu" "$HOME/.local/share/Steam/compatibilitytools.d"
- name: Test winetricks
run: |
source .venv/bin/activate
sh tests/test_winetricks.sh
rm -rf "$HOME/.local/share/umu" "$HOME/.local/share/Steam/compatibilitytools.d"
- name: Test configuration file
run: |
source .venv/bin/activate
sh tests/test_config.sh
35 changes: 20 additions & 15 deletions .github/workflows/static.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ name: mypy

on:
push:
branches: [ "main" ]
branches: ["main"]
pull_request:
branches: [ "main" ]
branches: ["main"]

permissions:
contents: read
Expand All @@ -13,20 +13,25 @@ jobs:
build:
strategy:
matrix:
version: ["3.10"]
version: ["3.11"]

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.version }}
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
- name: Check types with mypy
run: |
pip install mypy
mypy .
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.version }}
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip uv mypy
- name: Setup venv
run: |
uv venv --python 3.11
source .venv/bin/activate
uv pip install -r requirements.in
- name: Check types with mypy
run: |
source .venv/bin/activate
mypy --python-version 3.11 .
58 changes: 35 additions & 23 deletions .github/workflows/umu-python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ name: umu-launcher

on:
push:
branches: [ "main" ]
branches: ["main"]
pull_request:
branches: [ "main" ]
branches: ["main"]

permissions:
contents: read
Expand All @@ -15,28 +15,40 @@ jobs:
matrix:
# tomllib requires Python 3.11
# Ubuntu latest (Jammy) provides Python 3.10
version: ["3.10", "3.11", "3.12", "3.13"]
version: ["3.11", "3.12", "3.13"]

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.version }}
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
pip install ruff
pip install python-xlib
pip install filelock
- name: Lint umu_*.py files with Ruff
run: |
pip install ruff
ruff check --output-format github ./umu/umu_*.py
- name: Test with unittest
run: python3 ./umu/umu_test.py
- name: Test with unittest for plugins
if: ${{ matrix.version == '3.11' || matrix.version == '3.12' || matrix.version == '3.13' }}
run: python3 ./umu/umu_test_plugins.py
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.version }}
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
pip install ruff
pip install python-xlib
pip install filelock
pip install uv
- name: Lint umu_*.py files with Ruff
run: |
pip install ruff
ruff check --output-format github ./umu/umu_*.py
- name: Setup venv
run: |
uv venv --python $pyver
source .venv/bin/activate
uv pip install -r requirements.in
env:
pyver: ${{ matrix.version }}
- name: Test with unittest
run: |
source .venv/bin/activate
python3 ./umu/umu_test.py
- name: Test with unittest for plugins
if: ${{ matrix.version == '3.11' || matrix.version == '3.12' || matrix.version == '3.13' }}
run: |
source .venv/bin/activate
python3 ./umu/umu_test_plugins.py
27 changes: 12 additions & 15 deletions Makefile.in
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ FLATPAK ?= xfalse

.PHONY: all
ifeq ($(FLATPAK), xtrue)
all: umu-dist umu-launcher
all: umu-dist umu-launcher umu-vendored
endif

.PHONY: install
Expand All @@ -30,8 +30,8 @@ SOURCE_DATE_EPOCH = $(shell LC_ALL=C date --date='@1580601600')
all: zipapp
install: zipapp-install
else
all: umu-dist umu-docs umu-launcher
install: umu-install umu-launcher-install
all: umu-dist umu-docs umu-launcher umu-vendored
install: umu-install umu-launcher-install umu-vendored-install
endif


Expand Down Expand Up @@ -99,20 +99,17 @@ umu-launcher-dist-install:
umu-launcher-install: umu-launcher-dist-install umu-launcher-bin-install


$(OBJDIR)/.build-umu-subprojects: | $(OBJDIR)
$(info :: Building subprojects )
pip3 install -r requirements.in -t $(OBJDIR)
$(OBJDIR)/.build-umu-vendored: | $(OBJDIR)
$(info :: Building vendored dependencies )
python3 -m pip install urllib3 -t $(OBJDIR)

.PHONY: umu-subprojects
umu-subprojects: $(OBJDIR)/.build-umu-subprojects
.PHONY: umu-vendored
umu-vendored: $(OBJDIR)/.build-umu-vendored

umu-subprojects-install:
umu-vendored-install: umu-vendored
$(info :: Installing subprojects )
install -d $(DESTDIR)$(PYTHONDIR)
cp -r $(OBJDIR)/*-info $(DESTDIR)$(PYTHONDIR)
cp -r $(OBJDIR)/Xlib $(DESTDIR)$(PYTHONDIR)
cp -r $(OBJDIR)/filelock $(DESTDIR)$(PYTHONDIR)
cp $(OBJDIR)/six.py $(DESTDIR)$(PYTHONDIR)
install -d $(DESTDIR)$(PYTHONDIR)/umu/_vendor
cp -r $(OBJDIR)/urllib3 $(DESTDIR)$(PYTHONDIR)/umu/_vendor

$(OBJDIR):
@mkdir -p $(@)
Expand Down Expand Up @@ -153,7 +150,7 @@ ZIPAPP_VENV := $(OBJDIR)/zipapp_venv
$(OBJDIR)/.build-zipapp: | $(OBJDIR)
$(info :: Building umu-launcher as zipapp )
$(PYTHON_INTERPRETER) -m venv $(ZIPAPP_VENV)
. $(ZIPAPP_VENV)/bin/activate && python3 -m pip install -t "$(ZIPAPP_STAGING)" -U --no-compile .
. $(ZIPAPP_VENV)/bin/activate && python3 -m pip install -t "$(ZIPAPP_STAGING)" -U --no-compile . truststore
cp umu/__main__.py "$(ZIPAPP_STAGING)"
find "$(ZIPAPP_STAGING)" -exec touch -h -d "$(SOURCE_DATE_EPOCH)" {} +
. $(ZIPAPP_VENV)/bin/activate && python3 -m zipapp $(ZIPAPP_STAGING) -o $(ZIPAPP) -p "$(PYTHON_INTERPRETER)" -c
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Borderlands 3 from EGS store.

## Building

Building umu-launcher currently requires `bash`, `make`, and `scdoc` for distribution, as well as the following Python build tools: [build](https://github.com/pypa/build), [hatchling](https://github.com/pypa/hatch), and [installer](https://github.com/pypa/installer).
Building umu-launcher currently requires `bash`, `make`, and `scdoc` for distribution, as well as the following Python build tools: [build](https://github.com/pypa/build), [hatchling](https://github.com/pypa/hatch), [installer](https://github.com/pypa/installer), and [pip](https://github.com/pypa/pip).

To build umu-launcher, after downloading and extracting the source code from this repository, change into the newly extracted directory
```shell
Expand All @@ -95,7 +95,7 @@ Change the `--prefix` as fit for your distribution, for example `/usr/local`, or
Then run `make` to build. After a successful build the resulting files should be available in the `./builddir` directory


## Installing
## Installing

To install umu-launcher run the following command after completing the steps described above
```shell
Expand Down
1 change: 1 addition & 0 deletions packaging/deb/debian/control
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Build-Depends:
python3-hatchling,
python3-installer,
python3-build,
python3-pip,
Standards-Version: 4.6.2
Homepage: https://github.com/Open-Wine-Components/umu-launcher
Vcs-Browser: https://github.com/Open-Wine-Components/umu-launcher
Expand Down
1 change: 1 addition & 0 deletions packaging/deb/ubuntu/control
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Build-Depends:
python3-hatchling,
python3-installer,
python3-build,
python3-pip,
Standards-Version: 4.6.2
Homepage: https://github.com/Open-Wine-Components/umu-launcher
Vcs-Browser: https://github.com/Open-Wine-Components/umu-launcher
Expand Down
9 changes: 8 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,14 @@ classifiers = [
"Topic :: Software Development :: Libraries :: Python Modules",
]
urls = { repository = "https://github.com/Open-Wine-Components/umu-launcher" }
dependencies = ["python-xlib>=0.33", "filelock>=3.9.0"]
# Note: urllib3 is a vendored dependency. When using our Makefile, it will be
# installed automatically.
dependencies = ["python-xlib>=0.33", "filelock>=3.9.0", "urllib3>=2.0.0,<3.0.0"]

[project.optional-dependencies]
# Recommended
# For network requests, use the system's CA bundle instead of certifi's
cli = ["truststore"]

[project.scripts]
umu-run = "umu.__main__:main"
Expand Down
3 changes: 2 additions & 1 deletion requirements.in
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
python-xlib>=0.33
filelock>=3.15.4
filelock>=3.15.4
urllib3>=2.0.0,<3.0.0
8 changes: 8 additions & 0 deletions umu/__main__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
import os
import sys
from zipfile import is_zipfile

if not is_zipfile(os.path.dirname(__file__)): # noqa: PTH120
sys.path.insert(
0,
f"{os.path.dirname(os.path.realpath(__file__, strict=True))}/_vendor", # noqa: PTH120
)

from argparse import ArgumentParser, Namespace, RawTextHelpFormatter

from umu import __version__
Expand Down
Loading