dev: update dependencies, testing, packaging, and linting/formatting #685
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lindsay-stevens
force-pushed
the
pyxform-dev-deps-update
branch
2 times, most recently
from
28ae921
to
46edd80
Compare
lognaturel
approved these changes
Feb 1, 2024
amjadtaleb
reviewed
Feb 6, 2024
- config: old [setup.cfg, setup.py], new [pyproject.toml] - format/lint: old [black, flake8, isort], new [ruff] - build/publish: old [build, twine], new [flit] - updated github actions and readme accordingly - source changes according to new default formatter and linter rules
- nobody has changed these files since 2011, and they're very incomplete - the docs generated aren't published anywhere - the readme instructions don't work with python 3.8 - sphinx 1.0.7 not compatible with python 3+ - newer sphinx versions have an OS dependency on OpenSSL 1.1.1+ - updated readme to refer to useful resources and current docs state
- mostly error handling, unused variables
- mostly mutable classvars, unnecessary noqa comments, list unpacking
- unnecessary calls to list() or dict(), redundant comprehensions
- unnecessary placeholders, unnecessary range starts (0), dict.values() - re-order ruff calls since `check` doesn't auto-format
- collapsible else-ifs, collapsible multiple comparisons
- use of assert, use of ElementTree, unchecked URL scheme
- use type-able NamedTuple, raise accurate errors, log exception TBs
- UP009 utf8-encoding-declaration unnecessary for python 3
- UP031 printf-string-formatting replace percent-formats with f-strings
- nose 1.3.7 is very old and not using any advanced test features that would require a 3rd party test library like nose2 or pytest - default pattern is test_* for runner to pick up TestCases
- remove redundant mode/encoding args - simplify unittest import in test_bugs.py
- xls2json_backends.py: both XLS and XLSX open and read the respective files, do processing, then close the file. But if an error occurs during processing then the file is left open. A ResourceWarning is raised when python cleans up the file handle. The ResourceWarning is visible during tests because test libraries remove the normal suppression of warnings during test runs. So normally a user probably wouldn't see it. The fix is to make sure the workbook object and file are cleaned up and closed with a contextmanager + try-except block. - test_bugs.py: fixed test that had been broken for a while. It was not being executed by nose test runner for some reason. After converting tests to unittest, it ran and failed. After doing some git archaeology it seems that the 3rd warning that was expected was one about Google Sheets not supporting certain file names, but while that warning was later removed, the test was not updated. The 2nd is commented on. - fixed strange string concatentation on the same line e.g. "a" "b".
- nose tests were not running this test for a while. The changes are:
- secondary instances / itemsets replacing inline choices.
- for media itext with no translation, the value/form not output
- probable bug: custom attributes not applied to repeat template
(they appear in first repeat but not the jr:template).
- bump actions version numbers - update syntax for install, build, test
- either missing before, or mistakenly removed in commit f491ea8
lindsay-stevens
force-pushed
the
pyxform-dev-deps-update
branch
from
46edd80
to
b7c496a
Compare
lindsay-stevens
added a commit
to lindsay-stevens/pyxform
that referenced
this pull request
Feb 8, 2024
lindsay-stevens
force-pushed
the
pyxform-dev-deps-update
branch
from
34866a8
to
64903fd
Compare
|
@lognaturel thanks for the review! I'll wait for #694 to be decided before merging. |
|
Let's go ahead and merge! I think we're going to let #694 simmer for a bit. We can always release off a special release branch if needed. |
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overall this PR modernises pyxform and lints a swathe of code. Most of the commits are progressive application of linter rules, which hopefully makes it clearer what is being done and why - refer to
pyproject.tomlto see what rules were enabled in that commit.Changes summary:
nose1.3.7 (2015) to built-inunittest, which can do everything required by pyxform. This also revealed some testing issues which are fixed:nosewas suppressing warnings so these were not visible until switching to unittest.nosefor some unknown reason.nosefor some unknown reason.flitsincesetuptoolsis no longer recommended for python. Another option ishatchbut it looked like it might be a bit overkill and/or introduce OS dependency requirements just for package building.ruff, which has linter/formatter rules that encompass all the existing tools (black, flake8, isort) and many more. An initial batch of extra linting rule sets are enabled and applied.sphinxbaseddocsstub folder - not published anywhere and broken anywayWhy is this the best possible solution? Were any other approaches considered?
These changes support upgrading pyxform supported Python versions, since the tools replaced in this PR are either out of date already or will not be supported in future.
A lot of code in pyxform is/was crufty due to it's origins in Python 2, and as a natural result of having dozens of committers over many years. The increased linting moves the code towards a more coherent style that is more consistent with modern Python code. This should make contributions easier in future.
What are the regression risks?
The update of AttributeColumnsTest seems to have revealed a bug introduced by #380 for custom attributes and repeats (will backref this ticket). Nobody seems to have complained about this though so perhaps it's a low priority to fix.
The release publishing GitHub action may not be 100% right since I'm not sure there's a nice way to test that.
Does this change require updates to documentation? If so, please file an issue here and include the link below.
The code should be functionally equivalent for users. Where applicable, the
READMEfile has been updated.Before submitting this PR, please make sure you have:
testsnosetestsunittestand verified all tests passblack pyxform testsruff check pyxform tests&ruff format pyxform teststo lint and format code