Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github Actions とpre-commitにpoetry exportを適用 #716

Merged
merged 5 commits into from
Jul 18, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
name: test

env:
POETRY_VERSION: "1.3.1"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

未使用な気がします!
バージョンはrequirements.txtにて固定しているので、ここに書いちゃうとバージョン不一致になってしまうかもです。

on:
push:
pull_request:
Expand Down Expand Up @@ -37,6 +40,22 @@ jobs:
python -m pip install --upgrade pip setuptools wheel
python -m pip install -r requirements-test.txt

- name: Run poetry and check
run: |
poetry export --without-hashes -o requirements.txt.check
poetry export --without-hashes --with dev -o requirements-dev.txt.check
poetry export --without-hashes --with test -o requirements-test.txt.check
poetry export --without-hashes --with license -o requirements-license.txt.check

diff -q requirements.txt requirements.txt.check || \
diff -q requirements-dev.txt requirements-dev.txt.check || \
diff -q requirements-test.txt requirements-test.txt.check || \
diff -q requirements-license.txt requirements-license.txt.check > /dev/null
if [$? = 1]; then
Hiroshiba marked this conversation as resolved.
Show resolved Hide resolved
echo "poetry export has some diff"
exit 1
fi

- run: pysen run lint

- name: Run pytest and get coverage
Expand Down
20 changes: 20 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,21 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/python-poetry/poetry
rev: '1.3.1'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ここもバージョン不一致になっちゃう可能性がありますね!
まあさほど問題ないのですが、もしrequirements.txtをpip installしたときに入るpoetryを使えるとちょっとかっこいいかもとか思いました。
難しそうなら一旦このままでも!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pip install -r requirements.txtしてからpoetry exportを叩くように変更しました

hooks:
- id: poetry-export
name: poetry-export
args: ["--without-hashes", "-o", "requirements.txt" ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

実際にpoetry addしてpoetry exportしないままpushしてみたらちゃんとエラーが出ました!
ただエラーメッセージが若干不親切だなとちょっと思いました。

これ実際にエクスポートしているのでrequirements.txtファイルが置き換わるんですよね。
なので出来上がったファイルを再度コミットしてくださいというエラメッセージにするのはどうでしょう?
(そういうことってできそうでしょうか?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

poetry_export_all.pyがpre-commitの際に実行されるように修正しました
このスクリプトはdiffに差分があった場合、メッセージを出すようになっています
キャプチャ

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

なるほどです!!

ちょっと1点だけ、環境によってはpythonではなくpython3コマンドとしてPythonスクリプトを実行している人もいるはずで、そういう人にとってはエラーが出てしまうなと思いました!
どうにかできる解決策を調べてみたんですが、何か良い方法は見つかりませんでした・・・。
(シェルスクリプトを書くとかがあったのですが、これまたOSによっては動かないことがあるので)

特に初学者の方がpython3コマンドを使っていた場合、なぜエラーが出るのかわからずにちょっと戸惑ってしまうかもです。
poetry-exportがエラーになるのはpoetryを使えるぐらい熟練者の方なので、自力でなんとかできるかもしれない。
この辺りのバランスを取ると、もしかしたらpoetry-exportを実行するのがいいのかもとちょっと思いました。

(せっかく書いていただいたのに申し訳ないです 🙇 )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

>pythonではなくpython3
このパターンが確かにありましたね……
pythonコマンドから 実行バージョンを取得して、そのバージョンに応じたコマンドで poetry_export_all.py を呼び出すとかでしょうか
それかpoetry-exportは pre-commitのentryで呼び出してgit diffによる確認だけスクリプトを呼び出す形にするか
やりようはある気がするので探ってみます

Copy link
Contributor Author

@FujisakiEx FujisakiEx Jul 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pre-commitのentryで呼び出してgit diffによる確認だけスクリプトを呼び出す形

これで問題なさそうだったので poetry_export_check.py にリネームして、git diffだけPythonスクリプトで呼び出すようにしました
pythonコマンドがなくてpython3だけ入ってるということは考慮から外したのですが、問題ないでしょうか?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

なるほどです!! 提案ありがとうございます!!

まあでも、「poetry-exportによるエラーの理由が分からずに混乱する人の数」よりも、「python3コマンドを使っていてエラーが発生し理由が分からずに混乱する人の数」の方が多いかもですかね。。。
だとしたらpoetry-export-checkの方はない方がいいかもです 🙇‍♂️

どの環境でも動く方法があればという感じかなと・・・!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

差分のチェックは bash コマンドから poetry_export_check.shを実行する方針にしました

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bashもwindowsだと動かないんですよね・・・・・・・。

ChatGPT君に聞いてみた感じ、なんか良い感じの方法はなさそうでした。
https://chat.openai.com/share/94cee641-dfe0-483d-b9f3-84085899db20
なにかの言語で書いたものをバイナリにビルドして~とかが紹介されていますが、流石に・・・。

せっかくチャレンジしてくださったのに申し訳ないです 🙇

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Windows10からbashが正式サポートされた認識でした……
消しておきます!

- id: poetry-export
name: poetry-export-dev
args: ["--without-hashes", "--with", "dev", "-o", "requirements-dev.txt" ]
- id: poetry-export
name: poetry-export-test
args: ["--without-hashes", "--with", "test", "-o", "requirements-test.txt" ]
- id: poetry-export
name: poetry-export-license
args: ["--without-hashes", "--with", "license", "-o", "requirements-license.txt" ]
- repo: local
hooks:
- id: pysen-lint
Expand All @@ -10,3 +25,8 @@ repos:
types: [file, python]
stages: [push]
pass_filenames: false
- id: poetry-export-check
name: poetry-export-check
entry: python poetry_export_check.py
language: python
stages: [commit]
10 changes: 10 additions & 0 deletions poetry_export_check.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import subprocess


def main():
return subprocess.run(["git", "diff", "--exit-code", "--quiet"], shell=True)


if __name__ == "__main__":
if main().returncode == 1:
print("Please add requirement files")
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ isort = "^5.12.0"
mypy = "~0.991"
pytest = "^6.2.5"
coveralls = "^3.2.0"
poetry = "^1.3.1"

[tool.poetry.group.license.dependencies]
pip-licenses = "^4.2.0"
Expand Down
33 changes: 33 additions & 0 deletions requirements-test.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,50 +5,78 @@ asgiref==3.6.0 ; python_version >= "3.11" and python_version < "3.12"
atomicwrites==1.4.0 ; python_version >= "3.11" and python_version < "3.12" and sys_platform == "win32"
attrs==22.2.0 ; python_version >= "3.11" and python_version < "3.12"
black==22.12.0 ; python_version >= "3.11" and python_version < "3.12"
cachecontrol[filecache]==0.12.11 ; python_version >= "3.11" and python_version < "3.12"
certifi==2022.12.7 ; python_version >= "3.11" and python_version < "3.12"
cffi==1.15.1 ; python_version >= "3.11" and python_version < "3.12"
charset-normalizer==2.1.1 ; python_version >= "3.11" and python_version < "3.12"
cleo==2.0.1 ; python_version >= "3.11" and python_version < "3.12"
click==8.0.4 ; python_version >= "3.11" and python_version < "3.12"
colorama==0.4.4 ; python_version >= "3.11" and python_version < "3.12" and sys_platform == "win32" or python_version >= "3.11" and python_version < "3.12" and platform_system == "Windows"
colorlog==4.8.0 ; python_version >= "3.11" and python_version < "3.12"
coverage==5.5 ; python_version >= "3.11" and python_version < "3.12"
coveralls==3.2.0 ; python_version >= "3.11" and python_version < "3.12"
crashtest==0.4.1 ; python_version >= "3.11" and python_version < "3.12"
cryptography==39.0.0 ; python_version >= "3.11" and python_version < "3.12" and sys_platform == "linux"
cython==0.29.34 ; python_version >= "3.11" and python_version < "3.12"
dacite==1.7.0 ; python_version >= "3.11" and python_version < "3.12"
distlib==0.3.6 ; python_version >= "3.11" and python_version < "3.12"
docopt==0.6.2 ; python_version >= "3.11" and python_version < "3.12"
dulwich==0.20.50 ; python_version >= "3.11" and python_version < "3.12"
fastapi==0.70.0 ; python_version >= "3.11" and python_version < "3.12"
filelock==3.8.2 ; python_version >= "3.11" and python_version < "3.12"
flake8-bugbear==23.1.20 ; python_version >= "3.11" and python_version < "3.12"
flake8==6.0.0 ; python_version >= "3.11" and python_version < "3.12"
gitdb==4.0.10 ; python_version >= "3.11" and python_version < "3.12"
gitpython==3.1.29 ; python_version >= "3.11" and python_version < "3.12"
h11==0.14.0 ; python_version >= "3.11" and python_version < "3.12"
html5lib==1.1 ; python_version >= "3.11" and python_version < "3.12"
idna==3.4 ; python_version >= "3.11" and python_version < "3.12"
importlib-metadata==4.13.0 ; python_version >= "3.11" and python_version < "3.12"
iniconfig==1.1.1 ; python_version >= "3.11" and python_version < "3.12"
isort==5.12.0 ; python_version >= "3.11" and python_version < "3.12"
jaraco-classes==3.2.3 ; python_version >= "3.11" and python_version < "3.12"
jeepney==0.8.0 ; python_version >= "3.11" and python_version < "3.12" and sys_platform == "linux"
jinja2==3.1.2 ; python_version >= "3.11" and python_version < "3.12"
jsonschema==4.17.3 ; python_version >= "3.11" and python_version < "3.12"
keyring==23.13.1 ; python_version >= "3.11" and python_version < "3.12"
lockfile==0.12.2 ; python_version >= "3.11" and python_version < "3.12"
markupsafe==2.1.2 ; python_version >= "3.11" and python_version < "3.12"
mccabe==0.7.0 ; python_version >= "3.11" and python_version < "3.12"
more-itertools==9.0.0 ; python_version >= "3.11" and python_version < "3.12"
msgpack==1.0.4 ; python_version >= "3.11" and python_version < "3.12"
mypy-extensions==0.4.3 ; python_version >= "3.11" and python_version < "3.12"
mypy==0.991 ; python_version >= "3.11" and python_version < "3.12"
numpy==1.24.2 ; python_version >= "3.11" and python_version < "3.12"
packaging==22.0 ; python_version >= "3.11" and python_version < "3.12"
pathspec==0.10.3 ; python_version >= "3.11" and python_version < "3.12"
pexpect==4.8.0 ; python_version >= "3.11" and python_version < "3.12"
pkginfo==1.9.4 ; python_version >= "3.11" and python_version < "3.12"
platformdirs==2.6.2 ; python_version >= "3.11" and python_version < "3.12"
pluggy==1.0.0 ; python_version >= "3.11" and python_version < "3.12"
poetry-core==1.4.0 ; python_version >= "3.11" and python_version < "3.12"
poetry-plugin-export==1.2.0 ; python_version >= "3.11" and python_version < "3.12"
poetry==1.3.1 ; python_version >= "3.11" and python_version < "3.12"
ptyprocess==0.7.0 ; python_version >= "3.11" and python_version < "3.12"
py==1.11.0 ; python_version >= "3.11" and python_version < "3.12"
pycodestyle==2.10.0 ; python_version >= "3.11" and python_version < "3.12"
pycparser==2.21 ; python_version >= "3.11" and python_version < "3.12"
pydantic==1.10.2 ; python_version >= "3.11" and python_version < "3.12"
pyflakes==3.0.1 ; python_version >= "3.11" and python_version < "3.12"
pyopenjtalk @ git+https://github.com/VOICEVOX/pyopenjtalk@827a3fc5c7dda7bbe832c0c69da98e39cc8cb2c3 ; python_version >= "3.11" and python_version < "3.12"
pyrsistent==0.19.3 ; python_version >= "3.11" and python_version < "3.12"
pysen==0.10.3 ; python_version >= "3.11" and python_version < "3.12"
pytest==6.2.5 ; python_version >= "3.11" and python_version < "3.12"
python-multipart==0.0.5 ; python_version >= "3.11" and python_version < "3.12"
pywin32-ctypes==0.2.0 ; python_version >= "3.11" and python_version < "3.12" and sys_platform == "win32"
pyworld==0.3.2 ; python_version >= "3.11" and python_version < "3.12"
pyyaml==6.0 ; python_version >= "3.11" and python_version < "3.12"
rapidfuzz==2.13.7 ; python_version >= "3.11" and python_version < "3.12"
requests-toolbelt==0.10.1 ; python_version >= "3.11" and python_version < "3.12"
requests==2.28.1 ; python_version >= "3.11" and python_version < "3.12"
scipy==1.10.1 ; python_version >= "3.11" and python_version < "3.12"
secretstorage==3.3.3 ; python_version >= "3.11" and python_version < "3.12" and sys_platform == "linux"
semver==3.0.0 ; python_version >= "3.11" and python_version < "3.12"
shellingham==1.5.0.post1 ; python_version >= "3.11" and python_version < "3.12"
six==1.16.0 ; python_version >= "3.11" and python_version < "3.12"
smmap==5.0.0 ; python_version >= "3.11" and python_version < "3.12"
sniffio==1.3.0 ; python_version >= "3.11" and python_version < "3.12"
Expand All @@ -57,7 +85,12 @@ starlette==0.16.0 ; python_version >= "3.11" and python_version < "3.12"
toml==0.10.2 ; python_version >= "3.11" and python_version < "3.12"
tomlkit==0.11.6 ; python_version >= "3.11" and python_version < "3.12"
tqdm==4.64.1 ; python_version >= "3.11" and python_version < "3.12"
trove-classifiers==2022.12.22 ; python_version >= "3.11" and python_version < "3.12"
typing-extensions==4.4.0 ; python_version >= "3.11" and python_version < "3.12"
unidiff==0.7.4 ; python_version >= "3.11" and python_version < "3.12"
urllib3==1.26.13 ; python_version >= "3.11" and python_version < "3.12"
uvicorn==0.15.0 ; python_version >= "3.11" and python_version < "3.12"
virtualenv==20.17.1 ; python_version >= "3.11" and python_version < "3.12"
webencodings==0.5.1 ; python_version >= "3.11" and python_version < "3.12"
xattr==0.10.1 ; python_version >= "3.11" and python_version < "3.12" and sys_platform == "darwin"
zipp==3.11.0 ; python_version >= "3.11" and python_version < "3.12"