Skip to content

Commit 549d287

Browse files
artemruartyomkoMortimerp9
authored
fs2 update (#47)
* fs2 update * mutox with new fs2 * ff * sonar setup * black * lint version * ff * py3.10 in lint * ff * updated version * ff * ff * ff * fairseq2 0.3.0rc1 * fairseq2n source maybe? * move to uv for deps on fairseq2 + lint/format * pin torchaudio * update README * update mypy ci * fix mypy2 * mypy 3 * fix mypy --------- Co-authored-by: artyomko <artyomko@etinyclues.com> Co-authored-by: Pierre Andrews <628467+Mortimerp9@users.noreply.github.com>
1 parent a914bbd commit 549d287

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+3675
-612
lines changed

‎.github/actions/setup/action.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
runs:
2+
using: composite
3+
steps:
4+
- name: "Install UV"
5+
shell: bash
6+
run: |
7+
curl -LsSf https://astral.sh/uv/install.sh | sh
8+
- name: "Install libsndfile"
9+
shell: bash
10+
run: |
11+
sudo apt-get install libsndfile1 python3-distutils

‎.github/workflows/lint_and_test.yml

Lines changed: 38 additions & 109 deletions
Original file line numberDiff line numberDiff line change
@@ -1,123 +1,52 @@
1-
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
2-
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
3-
4-
name: Python package
1+
name: Lint Python Code
52

63
on:
4+
# Trigger the workflow on push to master or any pull request
75
push:
8-
branches: ["main"]
96
pull_request:
10-
branches: ["main"]
7+
branches:
8+
- main
119

1210
jobs:
13-
linter:
11+
lock_file:
1412
runs-on: ubuntu-latest
15-
strategy:
16-
fail-fast: false
17-
matrix:
18-
python-version: ["3.8"]
19-
2013
steps:
21-
- uses: actions/checkout@v3
22-
- name: Set up Python ${{ matrix.python-version }}
23-
uses: actions/setup-python@v3
24-
with:
25-
python-version: ${{ matrix.python-version }}
26-
cache: "pip"
27-
- name: Install dependencies
28-
run: |
29-
sudo apt-get install libsndfile1
30-
python -m pip install --upgrade pip
31-
pip install -r requirements.txt
32-
pip install -r requirements-dev.txt
33-
- name: Lint with flake8
34-
run: |
35-
# stop the build if there are Python syntax errors or undefined names
36-
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
37-
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
38-
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
39-
- name: isort
40-
run: isort --check --diff .
41-
- name: black
42-
run: black --check --diff .
43-
44-
mypy:
14+
- uses: actions/checkout@v4
15+
- uses: ./.github/actions/setup
16+
- run: uv lock --locked
17+
linting:
4518
runs-on: ubuntu-latest
46-
strategy:
47-
fail-fast: false
48-
matrix:
49-
python-version: ["3.8"]
19+
needs: [lock_file]
5020
steps:
51-
- uses: actions/checkout@v3
52-
- name: Set up Python ${{ matrix.python-version }}
53-
uses: actions/setup-python@v3
54-
with:
55-
python-version: ${{ matrix.python-version }}
56-
cache: "pip"
57-
- name: Install dependencies
58-
run: |
59-
sudo apt-get install libsndfile1
60-
python -m pip install --upgrade pip
61-
pip install -r requirements.txt
62-
pip install -r requirements-dev.txt
63-
- name: mypy
64-
run: mypy --install-types --non-interactive ./ --cache-dir=.mypy_cache/
65-
66-
unit_test:
21+
- uses: actions/checkout@v4
22+
- uses: ./.github/actions/setup
23+
- run: uvx ruff check .
24+
formatting:
6725
runs-on: ubuntu-latest
68-
timeout-minutes: 20
69-
strategy:
70-
fail-fast: false
71-
matrix:
72-
python-version: ["3.8"]
26+
needs: [lock_file]
7327
steps:
74-
- uses: actions/checkout@v3
75-
- name: Set up Python ${{ matrix.python-version }}
76-
uses: actions/setup-python@v3
77-
with:
78-
python-version: ${{ matrix.python-version }}
79-
cache: "pip"
80-
- name: Install dependencies
81-
run: |
82-
sudo apt-get install libsndfile1
83-
pip install -r requirements.txt
84-
pip install -r requirements-dev.txt
85-
- name: pytest_unit
86-
run: pytest -s -v tests/unit_tests/
87-
88-
integration_test:
28+
- uses: actions/checkout@v4
29+
- uses: ./.github/actions/setup
30+
- run: uvx ruff check --select I .
31+
- run: uvx ruff format --check .
32+
type_consistency:
8933
runs-on: ubuntu-latest
90-
strategy:
91-
fail-fast: false
92-
matrix:
93-
python-version: ["3.8"]
94-
timeout-minutes: 40
34+
needs: [lock_file]
9535
steps:
96-
- uses: actions/checkout@v3
97-
- name: Set up Python ${{ matrix.python-version }}
98-
uses: actions/setup-python@v3
99-
with:
100-
python-version: ${{ matrix.python-version }}
101-
cache: "pip"
102-
- name: Install dependencies
103-
run: |
104-
sudo apt-get install libsndfile1
105-
python -m pip install --upgrade pip
106-
pip install -r requirements.txt
107-
pip install -r requirements-dev.txt
108-
- name: Free Disk Space (Ubuntu)
109-
uses: jlumbroso/free-disk-space@main
110-
with:
111-
# this might remove tools that are actually needed,
112-
# if set to "true" but frees about 6 GB
113-
tool-cache: false
114-
115-
# all of these default to true, but feel free to set to
116-
# "false" if necessary for your workflow
117-
android: true
118-
dotnet: true
119-
haskell: true
120-
large-packages: true
121-
docker-images: true
122-
- name: pytest_integration
123-
run: pytest -s -v tests/integration_tests/ --ignore-glob='*sonar*.py'
36+
- uses: actions/checkout@v4
37+
- uses: ./.github/actions/setup
38+
- run: uvx mypy --install-types --non-interactive --ignore-missing-imports .
39+
tests:
40+
runs-on: ubuntu-latest
41+
needs: [lock_file]
42+
steps:
43+
- uses: actions/checkout@v4
44+
- uses: ./.github/actions/setup
45+
- run: uv run --extra cpu pytest -s -v tests/unit_tests/
46+
build:
47+
runs-on: ubuntu-latest
48+
needs: [lock_file, linting, formatting, type_consistency, tests]
49+
steps:
50+
- uses: actions/checkout@v4
51+
- uses: ./.github/actions/setup
52+
- run: uv build

‎.pre-commit-config.yaml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
repos:
2+
- repo: https://github.com/astral-sh/uv-pre-commit
3+
rev: 0.5.7
4+
hooks:
5+
- id: uv-lock
6+
- repo: https://github.com/astral-sh/ruff-pre-commit
7+
rev: v0.8.2
8+
hooks:
9+
# Lint
10+
- id: ruff
11+
args: [ --fix ]
12+
# sort imports
13+
- id: ruff
14+
args: ["check", "--select", "I", "--fix"]
15+
# format
16+
- id: ruff-format

‎README.md

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,14 +31,11 @@ The full list of supported languages (along with download links) can be found he
3131

3232
You can install SONAR with `pip install sonar-space`. Note that there is another `sonar` package on pip that IS NOT this project, make sure to use `sonar-space` in your dependencies.
3333

34-
If you want to install SONAR manually, you can install it localy. SONAR depends mainly on [Fairseq2](https://github.com/facebookresearch/fairseq2) and can be installed using (tested with `python=3.8`)
34+
SONAR depends on fairseq2 and torch/torchaudio, you will have to install the variant (cpu/cuda/...) that works for your environment manully.
3535

36-
```bash
37-
pip install --upgrade pip
38-
pip install -e .
39-
```
36+
Check [fairseq2 variants](https://github.com/facebookresearch/fairseq2?tab=readme-ov-file#variants) for possible variants. Note that SONAR currently relies on the release candidate for fairseq2 0.3.0 rc1.
4037

41-
If fairseq2 does not provide a build for your machine, check the readme of that project to build it locally.
38+
If [fairseq2](https://github.com/facebookresearch/fairseq2) does not provide a build for your machine, check the readme of that project to build it locally.
4239

4340
## Usage
4441
fairseq2 will automatically download models into your `$TORCH_HOME/hub` directory upon using the commands below.

0 commit comments

Comments
 (0)