Update to Python 3.12.x (#5736) #5951

dsotirho-ucsc · 2024-02-10T01:59:25Z

Connected issues: #5736

Checklist

Author

PR is a draft
Target branch is develop
Name of PR branch matches issues/<GitHub handle of author>/<issue#>-<slug>
On ZenHub, PR is connected to all issues it (partially) resolves
PR description links to connected issues
PR title matches¹ that of a connected issue _{or comment in PR explains why they're different}
PR title references all connected issues
For each connected issue, there is at least one commit whose title references that issue

Author (partiality)

Added p tag to titles of partial commits
Added partial label to PR _{or this PR completely resolves all connected issues}
All connected issues are resolved partially _{or this PR does not have the partial label}

¹ when the issue title describes a problem, the corresponding PR
title is Fix: followed by the issue title

Author (reindex, API changes)

Added r tag to commit title _{or this PR does not require reindexing}
Added reindex label to PR _{or this PR does not require reindexing}
PR and connected issue are labeled API _{or this PR does not modify a REST API}
Added a (A) tag to commit title for backwards (in)compatible changes _{or this PR does not modify a REST API}
Updated REST API version number in app.py _{or this PR does not modify a REST API}

Author (chains)

This PR is blocked by previous PR in the chain _{or this PR is not chained to another PR}
Added base label to the blocking PR _{or this PR is not chained to another PR}
Added chained label to this PR _{or this PR is not chained to another PR}

Author (upgrading deployments)

Documented upgrading of deployments in UPGRADING.rst _{or this PR does not require upgrading deployments}
Added u tag to commit title _{or this PR does not require upgrading deployments}
Added upgrade label to PR _{or this PR does not require upgrading deployments}

Author (operator tasks)

Added checklist items for additional operator tasks _{or this PR does not require additional tasks}

Author (hotfixes)

Added F tag to main commit title _{or this PR does not include permanent fix for a temporary hotfix}
Reverted the temporary hotfixes for any connected issues _{or the prod branch has no temporary hotfixes for any connected issues}

Author (before every review)

Rebased PR branch on develop, squashed old fixups
Ran make requirements_update _{or this PR does not touch requirements*.txt, common.mk, Makefile and Dockerfile}
Added R tag to commit title _{or this PR does not touch requirements*.txt}
Added reqs label to PR _{or this PR does not touch requirements*.txt}
make integration_test passes in personal deployment _{or this PR does not touch functionality that could break the IT}

Peer reviewer (after requesting changes)

Uncheck the Author (before every review) checklists.

Peer reviewer (after approval)

PR is not a draft
Ticket is in Review requested column
Requested review from system administrator
PR is assigned to system administrator

System administrator (after requesting changes)

Uncheck the before every review checklists. Update the N reviews label.

System administrator (after approval)

Actually approved the PR
Labeled connected issues as demo or no demo
Commented on connected issues about demo expectations _{or all connected issues are labeled no demo}
Decided if PR can be labeled no sandbox
PR title is appropriate as title of merge commit
N reviews label is accurate
Moved ticket to Approved column
PR is assigned to current operator

Operator (before pushing merge the commit)

Operator (chain shortening)

Changed the target branch of the blocked PR to develop _{or this PR is not labeled base}
Removed the chained label from the blocked PR _{or this PR is not labeled base}
Removed the blocking relationship from the blocked PR _{or this PR is not labeled base}
Removed the base label from this PR _{or this PR is not labeled base}

Operator (after pushing the merge commit)

¹ When pushing the merge commit is skipped due to the PR being
labelled no sandbox, the next build triggered by a PR whose merge commit is
pushed determines this checklist item.

Operator (reindex)

Operator

Added CL item to prod promotion PR:

Deployed prod.shared

PR is assigned to no one

Shorthand for review comments

L line is too long
W line wrapping is wrong
Q bad quotes
F other formatting problem

codecov · 2024-02-13T17:39:52Z

Codecov Report

Attention: Patch coverage is 82.35294% with 3 lines in your changes missing coverage. Please review.

Project coverage is 85.37%. Comparing base (319995f) to head (92489a0).

Files with missing lines	Patch %	Lines
...llatlas/data/metadata/helpers/schema_validation.py	85.71%	2 Missing ⚠️
test/integration_test.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #5951      +/-   ##
===========================================
- Coverage    85.38%   85.37%   -0.01%     
===========================================
  Files          155      155              
  Lines        20754    20764      +10     
===========================================
+ Hits         17720    17728       +8     
- Misses        3034     3036       +2

Flag	Coverage Δ
	`85.37% <82.35%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

coveralls · 2024-02-13T17:39:57Z

coverage: 85.395% (-0.003%) from 85.398%
when pulling 92489a0 on issues/dsotirho-ucsc/5736-update-python-to-3-12
into 319995f on develop.

hannes-ucsc · 2024-09-25T19:34:27Z

docker_images.json

-    "docker.io/library/python:3.11.10-slim-bullseye": {
-        "digest": "sha256:21e2cc29feabc9fa1335feb5efd98ebaf323a5963b8670a2212289b1db74cf73",
-        "mirror_digest": "sha256:ccb0c6383303002a6d15d2f52a24eb240f46a16f38984b04012a46c6a3b81560",
+    "docker.io/library/python:3.12.6-slim-bullseye": {


Looks like you forgot to address #5736 (comment) which is a pretty significant unforced error, since I specifically remember triaging that comment with you to make sure you're aware of it.

dsotirho-ucsc · 2024-09-26T01:07:43Z

5951_IT_2024-09-25.txt

achave11-ucsc · 2024-09-26T23:00:05Z

README.md

@@ -2319,10 +2319,10 @@ If you add a dependency on a package with native code, you need to build the
 wheel manually:

 ```
-(.venv) ~/workspace/hca/azul$ docker run -it -v ${project_root}/:/root/azul python:3.11.6-bullseye bash
+(.venv) ~/workspace/hca/azul$ docker run -it -v ${project_root}/:/root/azul python:3.12.6-slim-bullseye bash


#5736 (comment)

Fulfills transitive requirement of distutils, which Python 3.12 removed from the standard library

For Python 3.12 support

Resolves error during `make requirements`: × Building wheel for jq (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [1390 lines of output] running bdist_wheel running build running build_ext ... jq.c:12124:55: error: no member named 'ob_digit' in 'struct _longobject' const digit* digits = ((PyLongObject*)x)->ob_digit; ~~~~~~~~~~~~~~~~~~ ^ jq.c:13608:47: error: no member named 'ob_digit' in 'struct _longobject' const digit* digits = ((PyLongObject*)b)->ob_digit; ~~~~~~~~~~~~~~~~~~ ^ 41 warnings and 7 errors generated. error: command '/usr/bin/clang' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for jq

Resolves errors originating from flake8 failing to parse f-strings: E122 continuation line missing indentation or outdented E131 continuation line unaligned for hanging indent E202 whitespace before ')' E202 whitespace before ']' E202 whitespace before '}' E203 whitespace before ':' E221 multiple spaces before operator E222 multiple spaces after operator E225 missing whitespace around operator E231 missing whitespace after ',' E231 missing whitespace after ':' E231 missing whitespace after ';' E271 multiple spaces after keyword E272 multiple spaces before keyword E702 multiple statements on one line (semicolon) E713 test for membership should be 'not in' W604 backticks are deprecated, use 'repr()'

Resolves error during `make requirements_update`: 19.11 Building wheels for collected packages: fastavro 19.11 Building wheel for fastavro (pyproject.toml): started 36.96 Building wheel for fastavro (pyproject.toml): finished with status 'error' 36.97 error: subprocess-exited-with-error 36.97 36.97 × Building wheel for fastavro (pyproject.toml) did not run successfully. 36.97 │ exit code: 1 36.97 ╰─> [73 lines of output] 36.97 <string>:31: SyntaxWarning: invalid escape sequence '\(' 36.97 running bdist_wheel 36.97 running build … 36.97 gcc: internal compiler error: Illegal instruction signal terminated program as 36.97 Please submit a full bug report, 36.97 with preprocessed source if appropriate. 36.97 See <file:///usr/share/doc/gcc-10/README.Bugs> for instructions. 36.97 error: command '/usr/bin/gcc' failed with exit code 4 36.97 [end of output]

Resolves error during test: ERROR: test_openapi (integration_test.OpenAPIIntegrationTest.test_openapi) (component='service') ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/daniel/repo/azul1/test/integration_test.py", line 1856, in test_openapi validate_spec(spec) File "/Users/daniel/repo/azul1/.venv/lib/python3.12/site-packages/openapi_spec_validator/shortcuts.py", line 17, in validate_spec return validator.validate(spec, spec_url=spec_url) ... AttributeError: '_io.BufferedReader' object has no attribute 'decode'

Resolves DeprecationWarning during schema validation: Traceback (most recent call last): File "/Users/daniel/.pyenv/versions/3.12.6/lib/python3.12/unittest/suite.py", line 306, in _tearDownPreviousClass tearDownClass() AssertionError: ["{message : DeprecationWarning('Automatically retrieving remote references can be a security vulnerability and is discouraged by the JSON Schema specifications. Relying on this behavior is deprecated and will shortly become an error. If you are sure you want to remotely retrieve your reference and that it is safe to do so, you can find instructions for doing so via referencing.Registry in the referencing documentation (https://referencing.readthedocs.org).'), category : 'DeprecationWarning', filename : '/Users/daniel/repo/azul1/.venv/lib/python3.12/site-packages/jsonschema/exceptions.py', lineno : 475, line : None}", ...]

Resolves finding during `make pep8`: F842 local variable 'cs' is annotated but never used

Silences finding during `make pep8`: E122 continuation line missing indentation or outdented

Resolves error during test: AttributeError: 'not_called' is not a valid assertion. Use a spec for the mock if 'not_called' is meant to be an attribute.

dsotirho-ucsc · 2024-09-28T02:46:52Z

5951_IT_2024-09-27.txt

github-actions bot added the orange [process] Done by the Azul team label Feb 10, 2024

dsotirho-ucsc added reqs [process] PR includes commit requiring ``make requirements`` upgrade [process] PR includes commit requiring manual upgrade labels Feb 10, 2024

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5736-update-python-to-3-12 branch 4 times, most recently from b810ca2 to c09e7d4 Compare February 13, 2024 17:17

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5736-update-python-to-3-12 branch from c09e7d4 to e4c1675 Compare February 14, 2024 22:45

achave11-ucsc force-pushed the develop branch from ed4b5e1 to 858f6fb Compare February 15, 2024 21:38

hannes-ucsc force-pushed the develop branch 3 times, most recently from 0762297 to 3f7f6ce Compare May 28, 2024 06:58

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5736-update-python-to-3-12 branch from e4c1675 to 3ab9f81 Compare August 8, 2024 17:40

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5736-update-python-to-3-12 branch from 3ab9f81 to ca1c534 Compare August 19, 2024 17:03

hannes-ucsc force-pushed the develop branch from f54b36e to d769f27 Compare August 20, 2024 22:09

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5736-update-python-to-3-12 branch 2 times, most recently from c6c31f6 to 9dd4804 Compare August 27, 2024 18:25

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5736-update-python-to-3-12 branch 11 times, most recently from af1fdf3 to b2bb083 Compare September 20, 2024 19:59

hannes-ucsc requested changes Sep 25, 2024

View reviewed changes

hannes-ucsc added the 1 review [process] Lead requested changes once label Sep 25, 2024

hannes-ucsc removed their assignment Sep 25, 2024

dsotirho-ucsc dismissed achave11-ucsc’s stale review via 4e3b463 September 25, 2024 21:03

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5736-update-python-to-3-12 branch from d0aeae3 to 4e3b463 Compare September 25, 2024 21:03

dsotirho-ucsc requested a review from nadove-ucsc as a code owner September 25, 2024 21:03

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5736-update-python-to-3-12 branch 3 times, most recently from 1d3ad1a to 5fe3df4 Compare September 26, 2024 00:32

dsotirho-ucsc requested a review from hannes-ucsc September 26, 2024 01:07

dsotirho-ucsc assigned hannes-ucsc Sep 26, 2024

achave11-ucsc reviewed Sep 26, 2024

View reviewed changes

dsotirho-ucsc added 15 commits September 27, 2024 16:26

Update to Python 3.12.x (#5736)

0039720

Make requirements_update download binaries, not source packages

8268ebb

[R] Add setuptools as a requirement

a3f0e61

Fulfills transitive requirement of distutils, which Python 3.12 removed from the standard library

[R] Update azul-chalice to 1.31.2+13

28cf132

For Python 3.12 support

Remove unassigned type-hinted variable

a9e9b17

Resolves finding during `make pep8`: F842 local variable 'cs' is annotated but never used

Add noqa comment for flake8 bug

81913a7

Silences finding during `make pep8`: E122 continuation line missing indentation or outdented

Replace not_called assert statement with assert_not_called()

8cd9a31

Resolves error during test: AttributeError: 'not_called' is not a valid assertion. Use a spec for the mock if 'not_called' is meant to be an attribute.

fixup! Replace not_called assert statement with assert_not_called()

523633e

Ignore DeprecationWarning from datetime, add FIXME (#5953)

c57176f

[R] Ran make requirements_update

92489a0

dsotirho-ucsc force-pushed the issues/dsotirho-ucsc/5736-update-python-to-3-12 branch from 5fe3df4 to 92489a0 Compare September 28, 2024 01:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to Python 3.12.x (#5736) #5951

Update to Python 3.12.x (#5736) #5951

dsotirho-ucsc commented Feb 10, 2024 •

edited by achave11-ucsc

Loading

codecov bot commented Feb 13, 2024 •

edited

Loading

coveralls commented Feb 13, 2024 •

edited

Loading

hannes-ucsc Sep 25, 2024

dsotirho-ucsc commented Sep 26, 2024

achave11-ucsc Sep 26, 2024

dsotirho-ucsc commented Sep 28, 2024

Update to Python 3.12.x (#5736) #5951

Are you sure you want to change the base?

Update to Python 3.12.x (#5736) #5951

Conversation

dsotirho-ucsc commented Feb 10, 2024 • edited by achave11-ucsc Loading

Checklist

Author

Author (partiality)

Author (reindex, API changes)

Author (chains)

Author (upgrading deployments)

Author (operator tasks)

Author (hotfixes)

Author (before every review)

Peer reviewer (after requesting changes)

Peer reviewer (after approval)

System administrator (after requesting changes)

System administrator (after approval)

Operator (before pushing merge the commit)

Operator (chain shortening)

Operator (after pushing the merge commit)

Operator (reindex)

Operator

Shorthand for review comments

codecov bot commented Feb 13, 2024 • edited Loading

Codecov Report

coveralls commented Feb 13, 2024 • edited Loading

hannes-ucsc Sep 25, 2024

Choose a reason for hiding this comment

dsotirho-ucsc commented Sep 26, 2024

achave11-ucsc Sep 26, 2024

Choose a reason for hiding this comment

dsotirho-ucsc commented Sep 28, 2024

dsotirho-ucsc commented Feb 10, 2024 •

edited by achave11-ucsc

Loading

codecov bot commented Feb 13, 2024 •

edited

Loading

coveralls commented Feb 13, 2024 •

edited

Loading