Use poetry in CI

NicolasGrosjean · Sep 27, 2024 · 5f1c49d · 5f1c49d
1 parent daf1be5
commit 5f1c49d
Show file tree

Hide file tree

Showing 25 changed files with 131 additions and 98 deletions.
diff --git a/.github/workflows/flake8_test_coverage.yml b/.github/workflows/flake8_test_coverage.yml
@@ -8,32 +8,29 @@ on:
     branches: [ master ]
 
 jobs:
-  build:
-
+  ci:
     runs-on: ubuntu-latest
-
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python 3.10
-      uses: actions/setup-python@v3
+      uses: actions/setup-python@v5
       with:
         python-version: "3.10"
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        pip install pytest coverage genbadge[all]
-        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+    - name: Install poetry
+      uses: abatilo/actions-poetry@v3
+    - name: Install the project dependencies
+      run: poetry install
     - name: Lint with flake8
       run: |
         # stop the build if there are Python syntax errors or undefined names
-        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+        poetry run flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
         # exit-zero treats all errors as warnings
-        flake8 . --count --exit-zero --max-line-length=119 --statistics
+        poetry run flake8 . --count --exit-zero --max-line-length=119 --statistics
     - name: Coverage with pytest
       run: |
-        coverage run --omit "tests/*.py" -m pytest ./tests/ --junitxml=./report.xml -s
-        coverage report
-        coverage xml
+        poetry run coverage run --omit "tests/*.py" -m pytest ./tests/ --junitxml=./report.xml -s
+        poetry run coverage report
+        poetry run coverage xml
         # genbadge tests -i ./report.xml -s -o ./tests.svg
         # genbadge coverage -i ./coverage.xml -s -o ./coverage.svg
     # - name: Setup node

diff --git a/README.md b/README.md
@@ -4,12 +4,12 @@
 
 # Paradox localization utils
 
-> A set of tools to manipulate Paradox localization files particularly for translating 
+> A set of tools to manipulate Paradox localization files particularly for translating
 
 ## Terms
 
-* Source: It refers to the source language of your translation
-* Destination (shorten dest): It refers to the destination language of your translation
+- Source: It refers to the source language of your translation
+- Destination (shorten dest): It refers to the destination language of your translation
 
 Example : If I translate from English to French, the source directory refers to the directory of English localisation files.
 
@@ -27,7 +27,7 @@ poetry install
 
 <!-- [On Windows] Copy the file `coverage.bat` in the folder `.git/hooks/` and name it `pre-commit` (it is normal there is no `.bat`). -->
 
-Install pre-commit 
+Install pre-commit
 
 ```shell
 pre-commit install
@@ -49,7 +49,7 @@ Activate the conda environment
 activate paradox_loc
 ```
 
-Install Levenshtein package (only needed for *apply_diff_all*)
+Install Levenshtein package (only needed for _apply_diff_all_)
 
 ```shell
 conda install python-levenshtein
@@ -61,7 +61,7 @@ OR
 pip install python-levenshtein
 ```
 
-Install pandas package (only needed for *extract_existing_translation*)
+Install pandas package (only needed for _extract_existing_translation_)
 
 ```shell
 pip install pandas
@@ -71,7 +71,7 @@ pip install pandas
 
 <!-- [On Windows] Copy the file `coverage.bat` in the folder `.git/hooks/` and name it `pre-commit` (it is normal there is no `.bat`). -->
 
-Install pre-commit 
+Install pre-commit
 
 ```shell
 pip install pre-commit
@@ -86,13 +86,13 @@ pre-commit install
 > It also copy in the parent of the target directory an extract of the source localization and all files in target_dir.
 
 ```shell
-python src/update_source.py <source_dir> <target_dir> <source_lang>
+python paradox_localization_utils/update_source.py <source_dir> <target_dir> <source_lang>
 ```
 
 Example to update the English files of the HoI4 mod Kaiserreich
 
 ```shell
-python src/update_source.py <...>\Steam\steamapps\workshop\content\394360\1521695605 <...>\GitHub\Traduction-FR-Kaiserreich\Traduction-FR-Kaiserreich english
+python paradox_localization_utils/update_source.py <...>\Steam\steamapps\workshop\content\394360\1521695605 <...>\GitHub\Traduction-FR-Kaiserreich\Traduction-FR-Kaiserreich english
 ```
 
 ### Update Paratranz
@@ -104,13 +104,13 @@ Get your Paratranz API token in the settings tab of your Paratranz profile.
 Get the Paratranz project id by open the project and see the number in the url.
 
 ```shell
-python src/update_paratranz.py <token> <project_id> <loc_dir> <language>
+python paradox_localization_utils/update_paratranz.py <token> <project_id> <loc_dir> <language>
 ```
 
 Example to update the RICE project (translation from English) on Paratranz
 
 ```shell
-python src/update_paratranz.py <token> 2617 <...>\GitHub\RICE-for-CK3\RICE\localization english
+python paradox_localization_utils/update_paratranz.py <token> 2617 <...>\GitHub\RICE-for-CK3\RICE\localization english
 ```
 
 To compile the code, run the following command
@@ -130,7 +130,7 @@ Get your Paratranz API token in the settings tab of your Paratranz profile.
 Get the Paratranz project id by open the project and see the number in the url.
 
 ```shell
-python src/download_paratranz.py <token> <project_id> <raw_dir_path> <utf8_dir_path>
+python paradox_localization_utils/download_paratranz.py <token> <project_id> <raw_dir_path> <utf8_dir_path>
 ```
 
 ### Add missing lines
@@ -140,61 +140,60 @@ python src/download_paratranz.py <token> <project_id> <raw_dir_path> <utf8_dir_p
 #### For Imperator Rome and sooner
 
 ```shell
-python src/add_missing_lines.py <source_dir> <dest_dir>
+python paradox_localization_utils/add_missing_lines.py <source_dir> <dest_dir>
 ```
 
 Example to add in French files missing lines from English files
 
 ```shell
-python src/add_missing_lines.py "<...>\game\localization\english" "<...>\game\localization\french"
+python paradox_localization_utils/add_missing_lines.py "<...>\game\localization\english" "<...>\game\localization\french"
 ```
 
 #### For EUIV, HoI4 or Stellaris
 
 ```shell
-python src/add_missing_lines.py <localisation_dir> <localisation_dir> -source_lang <source_lang> -dest_lang <dest_lang>
+python paradox_localization_utils/add_missing_lines.py <localisation_dir> <localisation_dir> -source_lang <source_lang> -dest_lang <dest_lang>
 ```
 
 Example to add in French files missing lines from English files
 
 ```shell
-python src/add_missing_lines.py "<...>\localisation" "<...>\localisation" -source_lang english -dest_lang french
+python paradox_localization_utils/add_missing_lines.py "<...>\localisation" "<...>\localisation" -source_lang english -dest_lang french
 ```
 
-
 ### Apply diff all
 
 > Apply the differences between an old source version and a current source version to your current destination files.
 
 #### For Imperator Rome and sooner, and new EUIV, HoI4 or Stellaris localisation format (a subdirectory for each language)
 
 ```shell
-python src/apply_diff_all.py <old_localisation_dir> <current_localisation_dir> 
+python paradox_localization_utils/apply_diff_all.py <old_localisation_dir> <current_localisation_dir>
 ```
 
-* <old_localisation_dir> contains the localisation of the old source version
-* <current_source_localisation_dir> contains the localisation of the new source version
-* <current_dest_localisation_dir> contains the localisation of the old destination version
+- <old_localisation_dir> contains the localisation of the old source version
+- <current_source_localisation_dir> contains the localisation of the new source version
+- <current_dest_localisation_dir> contains the localisation of the old destination version
 
 Example to apply in French files the modifications done in English
 
 ```shell
-python src/apply_diff_all.py "<...>\V1\localisation\english" "<...>\V2\localisation\english" -dest_dir "<...>\V1\localisation\french"
+python paradox_localization_utils/apply_diff_all.py "<...>\V1\localisation\english" "<...>\V2\localisation\english" -dest_dir "<...>\V1\localisation\french"
 ```
 
 #### For old EUIV, HoI4 or Stellaris localisation format (all files in the same directory)
 
 ```shell
-python src/apply_diff_all.py <old_localisation_dir> <current_localisation_dir> -source_lang <source_lang> -dest_lang <dest_lang>
+python paradox_localization_utils/apply_diff_all.py <old_localisation_dir> <current_localisation_dir> -source_lang <source_lang> -dest_lang <dest_lang>
 ```
 
-* <old_localisation_dir> contains the localisation of the old source version
-* <current_localisation_dir> contains the localisation of both new source version and old destination version
+- <old_localisation_dir> contains the localisation of the old source version
+- <current_localisation_dir> contains the localisation of both new source version and old destination version
 
 Example to apply in French files the modifications done in English
 
 ```shell
-python src/apply_diff_all.py "<...>\V1\localisation" "<...>\V2\localisation" -source_lang english -dest_lang french
+python paradox_localization_utils/apply_diff_all.py "<...>\V1\localisation" "<...>\V2\localisation" -source_lang english -dest_lang french
 ```
 
 #### Both format
@@ -206,7 +205,7 @@ The new and edited lines contains `:9 "`.
 If there is less than 10 modifications (according Levenshtein distance),
 the destination text is kept.
 
-All the lines to translate are exported in to_translate/file_to_translate_<source_lang>.yml.
+All the lines to translate are exported in to*translate/file_to_translate*<source_lang>.yml.
 
 We can also add the following parameter to the command line -keys_to_ignore <path_to_file_with_a_key_a_line>,
 to simply copy-paste lines with these keys.
@@ -217,43 +216,45 @@ It is useful when you have keys like spellcheck_ignore in the source files.
 > Extract existing translation from another game or mod to apply to your translation
 
 #### Limits
+
 The target is EUIV or sooner whereas the source can be CK2/Vic2 or EUIV and sooner.
 
 #### Usage
 
 ```shell
-python src/extract_existing_translation.py <extract_source_dir> <extract_dest_dir> <target_source_dir> <target_dest_dir> <source_lang> <dest_lang> -source_col_ck2 <source_col_ck2> -dest_col_ck2 <dest_col_ck2>
+python paradox_localization_utils/extract_existing_translation.py <extract_source_dir> <extract_dest_dir> <target_source_dir> <target_dest_dir> <source_lang> <dest_lang> -source_col_ck2 <source_col_ck2> -dest_col_ck2 <dest_col_ck2>
 ```
 
 Example to add in French CK3 files some texts from CK2 files.
 
 ```shell
-python src/extract_existing_translation.py "<...>\CK2\localisation" "<...>\CK2\localisation" "<...>\CK3\localization\english" "<...>\CK3\localization\french" english french -source_col_ck2 1 -dest_col_ck2 2
+python paradox_localization_utils/extract_existing_translation.py "<...>\CK2\localisation" "<...>\CK2\localisation" "<...>\CK3\localization\english" "<...>\CK3\localization\french" english french -source_col_ck2 1 -dest_col_ck2 2
 ```
 
 Example to add in French HoI IV mod files some texts from Vanilla files.
 
 ```shell
-python src/extract_existing_translation.py "<...>\Steam\steamapps\common\Hearts of Iron IV\localisation" "<...>\Steam\steamapps\common\Hearts of Iron IV\localisation" "<...>\mod\localisation" "<...>\mod\localisation" english french
+python paradox_localization_utils/extract_existing_translation.py "<...>\Steam\steamapps\common\Hearts of Iron IV\localisation" "<...>\Steam\steamapps\common\Hearts of Iron IV\localisation" "<...>\mod\localisation" "<...>\mod\localisation" english french
 ```
 
 ### Extract Paratranz translation
 
 > Extract translations from JSON files downloaded from [Paratranz](https://paratranz.cn) to apply to your translation.
 
 #### Limits
+
 This script only works for EUIV and sooner.
 
 #### Usage
 
 ```shell
-python src/extract_paratranz_translation.py <paratranz_dir> <localisation_dir> <language>
+python paradox_localization_utils/extract_paratranz_translation.py <paratranz_dir> <localisation_dir> <language>
 ```
 
 Example to extract french reviewed translations
 
 ```shell
-python src/extract_paratranz_translation.py "<...>\paratranz" "<...>\localisation" french
+python paradox_localization_utils/extract_paratranz_translation.py "<...>\paratranz" "<...>\localisation" french
 ```
 
 By default it extracts only reviewed translation.
@@ -262,75 +263,77 @@ If you want to extract not reviewed translation, add `-extract_not_review` to th
 Example to extract french translations (reviewed or not)
 
 ```shell
-python src/extract_paratranz_translation.py "<...>\paratranz" "<...>\localisation" french -extract_not_review
+python paradox_localization_utils/extract_paratranz_translation.py "<...>\paratranz" "<...>\localisation" french -extract_not_review
 ```
 
 ### Copy localization in other languages
 
 > Copy the localization of a language in other languages to allow to run your game/mod with this language with source texts instead of keys.
 
 #### Limits
+
 It works only for Imperator Rome and sooner.
 
 #### Basic Usage [Windows only]
+
 The usage is explicated in this [Steam guide](https://steamcommunity.com/sharedfiles/filedetails/?id=2342385980).
 
 #### Developer Usage
 
 ```shell
-python src/copy_on_other_languages.py <localization_dir> <source_lang> <dest_lang1> <dest_lang2> ... 
+python paradox_localization_utils/copy_on_other_languages.py <localization_dir> <source_lang> <dest_lang1> <dest_lang2> ...
 ```
 
-Example to add some languages to available only in English and French (the added languages will have English text). 
+Example to add some languages to available only in English and French (the added languages will have English text).
 
 ```shell
-python src/copy_on_other_languages.py "<...>\MyMod\localisation" english korean german russian simp_chinese spanish
+python paradox_localization_utils/copy_on_other_languages.py "<...>\MyMod\localisation" english korean german russian simp_chinese spanish
 ```
 
 To compile it (after running once `pip install cx_Freeze`), use the following command
 
 ```shell
-cxfreeze -c src/copy_on_other_languages.py
+cxfreeze -c paradox_localization_utils/copy_on_other_languages.py
 ```
- 
+
 ### Get duplicates keys
 
 > Get the list of duplicated keys in the localization to avoid localization erased by others.
- 
+
 #### Limits
- 
+
 The duplicated keys are searched in English localisation files only.
- 
+
 #### Usage
- 
+
 To get the full list of duplicated keys run the following command
 
 ```shell
-python src/get_duplicates_key.py "<...>\MyMod\localisation"
+python paradox_localization_utils/get_duplicates_key.py "<...>\MyMod\localisation"
 ```
- 
+
 To get only the list of duplicated keys which already cause a bug
-(the value is not the same for each duplication), add *-only_different_value* to the command.
-
-
+(the value is not the same for each duplication), add _-only_different_value_ to the command.
+
 ### [DEPRECATED] Apply diff for EUIV and sooner
 
 ```shell
-python src/apply_diff.py <old_source_file> <new_source_file> <dest_file>
+python paradox_localization_utils/apply_diff.py <old_source_file> <new_source_file> <dest_file>
 ```
 
 The destination file is override so don't hesitate to have a backup before running this.
 The new and edited lines contains `:9 "`
 
 Optionnal parameters :
-* ```-space_prefix <value>``` : Prefix (generally spaces) before key in files
-* ```-keep_edited``` : Keep destination value in case of edition.
-The default behavior replace the destination value by the new source value.
+
+- `-space_prefix <value>` : Prefix (generally spaces) before key in files
+- `-keep_edited` : Keep destination value in case of edition.
+  The default behavior replace the destination value by the new source value.
 
 Example to apply the English update from 0.11.1 to 0.11.2 to Russia file in French:
 
 ```shell
-python src/apply_diff.py <...>\0.11.1\localisation\KR_Russia_l_english.yml <...>\0.11.2\localisation\KR_Russia_l_english.yml <...>\Translation_FR\localisation\KR_Russia_l_french.yml -space_prefix "  " -keep_edited
+python paradox_localization_utils/apply_diff.py <...>\0.11.1\localisation\KR_Russia_l_english.yml <...>\0.11.2\localisation\KR_Russia_l_english.yml <...>\Translation_FR\localisation\KR_Russia_l_french.yml -space_prefix "  " -keep_edited
 ```
 
 After running this, the new and edited lines need to be fixed by hand by seeking `:9 "`
@@ -343,8 +346,7 @@ Run the tests with pytest
 pytest
 ```
 
-If some tests fails with error `ModuleNotFoundError: No module named 'src'`, run the following command in the root directory.
-
+If some tests fails with error `ModuleNotFoundError: No module named 'paradox_localization_utils'`, run the following command in the root directory.
 
 On Windows:
 

diff --git a/compile_update_paratranz.py b/compile_update_paratranz.py
@@ -7,5 +7,5 @@
     version="0.1",
     description="Update translation on Paratranz",
     options={"build_exe": build_exe_options},
-    executables=[Executable("src/update_paratranz.py")],
+    executables=[Executable("paradox_localization_utils/update_paratranz.py")],
 )
diff --git a/copy_on_other_languages/README.txt b/copy_on_other_languages/README.txt
@@ -1,5 +1,5 @@
 Usage guide : https://steamcommunity.com/sharedfiles/filedetails/?id=2342385980
 
-Source code : https://github.com/NicolasGrosjean/Paradox_localization_utils/blob/master/src/copy_on_other_languages.py
+Source code : https://github.com/NicolasGrosjean/Paradox_localization_utils/blob/master/paradox_localization_utils/copy_on_other_languages.py
 
 Licence MIT : https://github.com/NicolasGrosjean/Paradox_localization_utils/blob/master/Licence.md
diff --git a/paradox_localization_utils/__init__.py b/paradox_localization_utils/__init__.py
@@ -0,0 +1 @@
+