-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
python dependencies are affecting the CI of multiple CKF repositories #648
Comments
Because some dependencies of jsonschema now require the rust toolchain (see 1 and 2), building charms that depend on SDI (and therefore pull jsonschema) fail when building because none of them have rustc available when pip installing the charm's python dependencies. Reasoning: jsonschema 4.18.x introduced a dependency on rdps-py which at the same time introduced a dependency on maturin, a python package that requires the rustc toolchain to be built and installed from source. jsonschema 4.17.x does not have that dependency. Links: [1]python-jsonschema/jsonschema#1117 [2] python-jsonschema/jsonschema#1114 Partially fixes: canonical/bundle-kubeflow#648
While reseaching the solution for Dex Auth requirements update (PR )it was confirmed that the above two solutions proposed in this issue are not applicable. |
@i-chvets we have a new version of SDI that should be compatible with |
This commit updates the CI with standard CI configuration (juju, actions, deps) that is shared across multiple repositories. All python dependencies are also updated to their latest working versions. This PR also includes library updates. Finally, some formatting was applied on non-compliant files (copyright, style). This change also contains a fix for #84, an issue caused by a missing toolchain in the charm that allows building certain python packages. For more information, please refer to canonical/bundle-kubeflow#648. Fixes #84 Part of canonical/bundle-kubeflow#648
#136 Summary of changes: - Pinned SDDI to 0.3.6 that has a fix. Details are in canonical/bundle-kubeflow#648
It worked like a charm. Fix is ready for review canonical/dex-auth-operator#143 |
This commit updates the CI with standard CI configuration (juju, actions, deps) that is shared across multiple repositories. All python dependencies are also updated to their latest working versions. This PR also includes library updates. Finally, some formatting was applied on non-compliant files (copyright, style). This change also contains a fix for #147, an issue caused by a missing toolchain in the charm that allows building certain python packages. For more information, please refer to canonical/bundle-kubeflow#648. Fixes #141 Fixes #147 Part of canonical/bundle-kubeflow#648 * test: Fix assertion fails in unit tests Fix assertion errors raised in unit tests caused by updating ops from 1.5 to 2+. Due to this change, we now have to take an extra step to enable container connectivity when not using `begin_with_initial_hooks`. Closes #141 Signed-off-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com> --------- Signed-off-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com> Co-authored-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
* ci, build: update CI and python dependencies This commit updates the CI with standard CI configuration (juju, actions, deps) that is shared across multiple repositories. All python dependencies are also updated to their latest working versions. This PR also includes library updates. Finally, some formatting was applied on non-compliant files (copyright, style). This change also contains a fix for #101 , an issue caused by a missing toolchain in the charm that allows building certain python packages. For more information, please refer to canonical/bundle-kubeflow#648. Fixes #101 Part of canonical/bundle-kubeflow#648 --------- Signed-off-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com> Co-authored-by: Phoevos Kalemkeris <phoevos.kalemkeris@canonical.com>
This commit updates the CI with standard CI configuration (juju, actions, deps) that is shared across multiple repositories. All python dependencies are also updated to their latest working versions. This PR also includes library updates. Finally, some formatting was applied on non-compliant files (copyright, style). Fixes #274, #275 Part of canonical/bundle-kubeflow#648
* fix: use new sdi 0.3.6 #136 Summary of changes: - Pinned SDDI to 0.3.6 that has a fix. Details are in canonical/bundle-kubeflow#648 - update integrate - Pinned juju channel. - Updated integration requirements - Updated integation workflow - Pinned ops - Added comment
CI and integration tests were broken due to dependency update and admissiob-webhook update in latest/edge. Summary of changes: - Added trust to admission webhook deployment in test. - Updated requirements as per solutions discussed in canonical/bundle-kubeflow#648
* build: bump juju 2.9.44.0, sdi->0.7.0, pyyaml->6.0.1 Part of canonical/bundle-kubeflow#648
Merge a collection of chores and fixes from dnplas-dev-fix-ci into main. All commits in this PR have been already reviewed, in their respective PRs. Changes: * fix: add ml-pipeline-service.yaml.j2 in K8S_RESOURCE_FILES (#255) (#279) * ci, tests: fix testing in GH runners for kfp-api individual integration tests (#273) * build: bump juju 2.9.44.0, sdi->0.7.0, pyyaml->6.0.1 (#290) * ci, tests: fix testing in GH runners for kfp-api individual integration tests (#273) (#292) * ci, build: use action to free space from GH runners and update dependencies (#293) Part of canonical/bundle-kubeflow#648
* build: bump juju 2.9.44.0, sdi->0.7.0, pyyaml->6.0.1 Part of canonical/bundle-kubeflow#648 * build: unpin setuptools and pip
* build: bump juju 2.9.44.0, sdi->0.7.0, pyyaml->6.0.1 Part of canonical/bundle-kubeflow#648 * ci: install juju from channel 2.9/stable
* build: bump juju 2.9.44.0, sdi->0.7.0, pyyaml->6.0.1 Part of canonical/bundle-kubeflow#648 * tests: bump istio-pilot 1.5->1.16 stable
Closing this issue as all the repositories handled by the MLOps team have now the changes suggested by this issue, these errors should not be present anymore. |
In the past few weeks, a number of charm and tests dependencies (python packages) have been affecting the health of most of the CIs of the charms that belong to the CKF bundle. We have identified two main issues:
pyyaml
6.0 fails to install and returnsAttributeError: cython_sources
Description
pyyaml
is a dependency ofpython-libjuju
, which before v2.9.44 pinnedpyyaml>=5.1.2,<=6.0
, which did not allowpyyaml 6.0.1
to be installed, preventing a fix for yaml/pyyaml#601 to be pulled in.For more details, please go here.
The issue can be identified when installing pyyaml, pip will throw the following message:
Pip install output
Affects
Any type of code that uses
python-libjuju=<2.9.43
, specially of course, integration tests.Solution
Bump the version of
python-libjuju>=2.9.44
, making sure thatpyyaml>6.0
is installed.Fail to build charm
RuntimeError: Failed to build charm
Description
Although this is a generic message when a
charmcraft pack
command fails, we are focusing on the following error message:charmcraft pack logs
This error is caused by a
jsonschema
v4.18.x dependency that requiresrustc
andcargo
to be present in the charm to be built. Please refer to python-jsonschema/jsonschema#1117 and python-jsonschema/jsonschema#1114 for more details on the issue.Affects
This error will be present ONLY when bumping
jsonschema
to 4.18.x in a charm'srequirements.txt
file.Solution
For the majority of the cases,
jsonschema
is introduced as a dependency ofSDI
, therefore using the latest version (0.7.0) of that package and updating therequirements.txt
file should solve thecharmcraft pack
issue.In the case this is happens outside of a charm that uses SDI, pinning
jsonschema
may fix the issue.The text was updated successfully, but these errors were encountered: