build: Add rust compiler to charms

[orfeas-k]

Context

Hit during #1156 and #1177, we hit the issue where charms fail to be built due to the absence of a rust compiler during build time.

• argo-controller
• kfp charms apart from metadata-writer
• knative charms
• kserve-controller
• kubeflow-profiles-operator
• kubeflow-tensorboard-operators
• volumes-operator
• metacontroller-operator
• minio-operator
• namespace-node-affinity-operator
• ngc-integrator
• resource-dispatcher
• training-operator

What needs to get done

Add appropriate dependencies as we previously did canonical/kubeflow-dashboard-operator@c31941b or follow what mysql-k8s team is doing seen here canonical/charmcraft#1722 (comment). The latter proved to be building faster the charm ( 12 minutes compared to 18 minutes for argo-controller)

Definition of Done

Charms can be built with dependencies requiring a rust compiler.

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-6645.

This message was autogenerated

[orfeas-k]

KFP

KFP tests fail due to space in the runner as seen here https://github.com/canonical/kfp-operators/actions/runs/12162787081/job/33958507596?pr=631#step:5:468 (same as canonical/kfp-operators#602)

FAILED tests/integration/test_kfp_functional_v2.py::test_create_and_monitor_run - AssertionError: assert 'FAILED' == 'SUCCEEDED'
  
  - SUCCEEDED
  + FAILED

and here https://github.com/canonical/kfp-operators/actions/runs/12162787081/job/33958506393?pr=631#step:5:442

unit-kfp-profile-controller-0: 08:30:35 ERROR juju.worker.uniter.operation hook "update-status" (via unknown/invalid hook handler) failed: starting jujuc server: listening to jujuc socket: mkdir /var/lib/juju/agents/unit-kfp-profile-controller-0/426642296: no space left on device

This means that adding rust compiler to the charms needs first this feature to land canonical/kfp-operators#627.

[orfeas-k]

Note that as done in mysql-k8s (see canonical/charmcraft#1722 (comment)), the build packages pkg-config, libffi-dev, libssl-dev are needed. Otherwise, building fails with the following error:

...
:: Running external command ['python3', '-m', 'venv', '/root/parts/charm/build/staging-venv']
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--force-reinstall', 'pip@https://files.pythonhosted.org/packages/c0/d0/9641dc7b05877874c6418f8034ddefc809495e65caa14d38c7551cd114bb/pip-24.1.1.tar.gz']
::    :: Collecting pip@ https://files.pythonhosted.org/packages/c0/d0/9641dc7b05877874c6418f8034ddefc809495e65caa14d38c7551cd114bb/pip-24.1.1.tar.gz
::    ::   Downloading pip-24.1.1.tar.gz (1.9 MB)
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::     Preparing wheel metadata: started
::    ::     Preparing wheel metadata: finished with status 'done'
::    :: Building wheels for collected packages: pip
::    ::   Building wheel for pip (PEP 517): started
::    ::   Building wheel for pip (PEP 517): finished with status 'done'
::    ::   Created wheel for pip: filename=pip-24.1.1-py3-none-any.whl size=1824282 sha256=e8294edf16c52b89a0eccd34036d30218ae3bdac7a2acf323b6dfe8d3d8e88e1
::    ::   Stored in directory: /root/.cache/pip/wheels/a2/93/80/a8fa207c102102e4c7f0ea7965db2509466086aa06a927a5ce
::    :: Successfully built pip
::    :: Installing collected packages: pip
::    ::   Attempting uninstall: pip
::    ::     Found existing installation: pip 20.0.2
::    ::     Uninstalling pip-20.0.2:
::    ::       Successfully uninstalled pip-20.0.2
::    :: Successfully installed pip-24.1.1
:: Installing Python pre-dependencies from source.
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', 'wheel', 'setuptools', 'pip']
::    :: Collecting wheel
::    ::   Downloading wheel-0.45.1.tar.gz (107 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 107.5/107.5 kB 19.5 MB/s eta 0:00:00
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Requirement already satisfied: setuptools in ./staging-venv/lib/python3.8/site-packages (44.0.0)
::    :: Requirement already satisfied: pip in ./staging-venv/lib/python3.8/site-packages (24.1.1)
::    :: Building wheels for collected packages: wheel
::    ::   Building wheel for wheel (pyproject.toml): started
::    ::   Building wheel for wheel (pyproject.toml): finished with status 'done'
::    ::   Created wheel for wheel: filename=wheel-0.45.1-py3-none-any.whl size=72494 sha256=708e7481cc80179af0e556bbf0cc00b8444c7321e2700b8d8580231d13017248
::    ::   Stored in directory: /root/.cache/pip/wheels/1b/00/e6/216671cc7f33efcb5fc4e2b16b86f829cc57d2e0e28d9b90e8
::    :: Successfully built wheel
::    :: Installing collected packages: wheel
::    :: Successfully installed wheel-0.45.1
::    ::
Notice: ::    :: [notice] A new release of pip is available: 24.1.1 -> 24.3.1
Notice: ::    :: [notice] To update, run: python3 -m pip install --upgrade pip
:: Installing packages from requirements files and charm lib dependencies.
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', '--requirement=requirements.txt']
::    :: Collecting annotated-types==0.7.0 (from -r requirements.txt (line 7))
::    ::   Downloading annotated_types-0.7.0.tar.gz (16 kB)
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting anyio==4.5.2 (from -r requirements.txt (line 9))
::    ::   Downloading anyio-4.5.2.tar.gz (171 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 171.3/171.3 kB 20.4 MB/s eta 0:00:00
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting attrs==24.2.0 (from -r requirements.txt (line 11))
::    ::   Downloading attrs-24.2.0.tar.gz (792 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 792.7/792.7 kB 96.0 MB/s eta 0:00:00
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting backports-strenum==1.3.1 (from -r requirements.txt (line 13))
::    ::   Downloading backports_strenum-1.3.1.tar.gz (7.3 kB)
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting bcrypt==4.2.1 (from -r requirements.txt (line 15))
::    ::   Downloading bcrypt-4.2.1.tar.gz (24 kB)
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting cachetools==5.5.0 (from -r requirements.txt (line 17))
::    ::   Downloading cachetools-5.5.0.tar.gz (27 kB)
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting certifi==2024.8.30 (from -r requirements.txt (line 19))
::    ::   Downloading certifi-2024.8.30.tar.gz (168 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 168.5/168.5 kB 40.9 MB/s eta 0:00:00
::    ::   Preparing metadata (setup.py): started
::    ::   Preparing metadata (setup.py): finished with status 'done'
::    :: Collecting cffi==1.17.1 (from -r requirements.txt (line 25))
::    ::   Downloading cffi-1.17.1.tar.gz (516 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 516.6/516.6 kB 53.3 MB/s eta 0:00:00
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting charmed-kubeflow-chisme==0.4.3 (from -r requirements.txt (line 29))
::    ::   Downloading charmed_kubeflow_chisme-0.4.3.tar.gz (43 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.3/43.3 kB 13.4 MB/s eta 0:00:00
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting charset-normalizer==3.4.0 (from -r requirements.txt (line 31))
::    ::   Downloading charset_normalizer-3.4.0.tar.gz (106 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 106.6/106.6 kB 29.3 MB/s eta 0:00:00
::    ::   Preparing metadata (setup.py): started
::    ::   Preparing metadata (setup.py): finished with status 'done'
::    :: Collecting cosl==0.0.45 (from -r requirements.txt (line 33))
::    ::   Downloading cosl-0.0.45.tar.gz (67 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.5/67.5 kB 16.9 MB/s eta 0:00:00
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: still running...
::    ::   Installing build dependencies: still running...
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting cryptography==44.0.0 (from -r requirements.txt (line 35))
::    ::   Downloading cryptography-44.0.0.tar.gz (710 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 710.7/710.7 kB 91.0 MB/s eta 0:00:00
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: still running...
::    ::   Installing build dependencies: still running...
::    ::   Installing build dependencies: finished with status 'error'
::    ::   error: subprocess-exited-with-error
::    ::
::    ::   × pip subprocess to install build dependencies did not run successfully.
::    ::   │ exit code: 1
::    ::   ╰─> [89 lines of output]
::    ::       Collecting maturin<2,>=1
::    ::         Downloading maturin-1.7.8.tar.gz (195 kB)
::    ::            ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 195.7/195.7 kB 2.5 MB/s eta 0:00:00
::    ::         Installing build dependencies: started
::    ::         Installing build dependencies: finished with status 'done'
::    ::         Getting requirements to build wheel: started
::    ::         Getting requirements to build wheel: finished with status 'done'
::    ::         Preparing metadata (pyproject.toml): started
::    ::         Preparing metadata (pyproject.toml): finished with status 'done'
::    ::       Collecting cffi>=1.12
::    ::         Using cached cffi-1.17.1.tar.gz (516 kB)
::    ::         Installing build dependencies: started
::    ::         Installing build dependencies: finished with status 'done'
::    ::         Getting requirements to build wheel: started
::    ::         Getting requirements to build wheel: finished with status 'done'
::    ::         Preparing metadata (pyproject.toml): started
::    ::         Preparing metadata (pyproject.toml): finished with status 'done'
::    ::       Collecting setuptools!=74.0.0,!=74.1.0,!=74.1.1,!=74.1.2
::    ::         Using cached setuptools-75.3.0-py3-none-any.whl
::    ::       Collecting tomli>=1.1.0 (from maturin<2,>=1)
::    ::         Using cached tomli-2.2.1-py3-none-any.whl
::    ::       Collecting pycparser (from cffi>=1.12)
::    ::         Downloading pycparser-2.22.tar.gz (172 kB)
::    ::            ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 172.7/172.7 kB 9.1 MB/s eta 0:00:00
::    ::         Preparing metadata (setup.py): started
::    ::         Preparing metadata (setup.py): finished with status 'done'
::    ::       Building wheels for collected packages: maturin, cffi, pycparser
::    ::         Building wheel for maturin (pyproject.toml): started
::    ::         Building wheel for maturin (pyproject.toml): still running...
::    ::         Building wheel for maturin (pyproject.toml): still running...
::    ::         Building wheel for maturin (pyproject.toml): finished with status 'done'
::    ::         Created wheel for maturin: filename=maturin-1.7.8-cp38-cp38-linux_x86_64.whl size=5995658 sha256=e9cbdec65db8641ca6bfcfc3c1f7c89ee91357d80a058e10f54fdbf57c44c[549](https://github.com/canonical/argo-operators/actions/runs/12199502801/job/34033534048#step:4:550)
::    ::         Stored in directory: /root/.cache/pip/wheels/f1/2a/cc/94408ea6aec8dad6addae6c4bb4fb04597715b4315e43aca2d
::    ::         Building wheel for cffi (pyproject.toml): started
::    ::         Building wheel for cffi (pyproject.toml): finished with status 'error'
::    ::         error: subprocess-exited-with-error
::    ::
::    ::         × Building wheel for cffi (pyproject.toml) did not run successfully.
::    ::         │ exit code: 1
::    ::         ╰─> [35 lines of output]
::    ::             running bdist_wheel
::    ::             running build
::    ::             running build_py
::    ::             creating build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/error.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/vengine_gen.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/recompiler.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/model.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/vengine_cpy.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/verifier.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/setuptools_ext.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/_shimmed_dist_utils.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/__init__.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/cffi_opcode.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/ffiplatform.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/cparser.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/backend_ctypes.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/lock.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/pkgconfig.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/api.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/_imp_emulation.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/commontypes.py -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/_cffi_include.h -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/parse_c_type.h -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/_embedding.h -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             copying src/cffi/_cffi_errors.h -> build/lib.linux-x86_64-cpython-38/cffi
::    ::             running build_ext
::    ::             building '_cffi_backend' extension
::    ::             creating build/temp.linux-x86_64-cpython-38/src/c
::    ::             x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/root/parts/charm/build/staging-venv/include -I/usr/include/python3.8 -c src/c/_cffi_backend.c -o build/temp.linux-x86_64-cpython-38/src/c/_cffi_backend.o
::    ::             src/c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
::    ::                15 | #include <ffi.h>
::    ::                   |          ^~~~~~~
::    ::             compilation terminated.
::    ::             error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
::    ::             [end of output]
::    ::
::    ::         note: This error originates from a subprocess, and is likely not a problem with pip.
::    ::         ERROR: Failed building wheel for cffi
::    ::         Building wheel for pycparser (setup.py): started
::    ::         Building wheel for pycparser (setup.py): finished with status 'done'
::    ::         Created wheel for pycparser: filename=pycparser-2.22-py3-none-any.whl size=117532 sha256=0c29a4cd2b8e5899651b460a8dfa20524f1a3ead0f26e7b24cbfee87b84bad9d
::    ::         Stored in directory: /root/.cache/pip/wheels/9d/ac/d9/e5fa4cb20ee9c45d0664af14d6a9da2664[558](https://github.com/canonical/argo-operators/actions/runs/12199502801/job/34033534048#step:4:559)3dc723e3ab868
::    ::       Successfully built maturin pycparser
::    ::       Failed to build cffi
::    ::       ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (cffi)
::    ::
Notice: ::    ::       [notice] A new release of pip is available: 24.1.1 -> 24.3.1
Notice: ::    ::       [notice] To update, run: python3 -m pip install --upgrade pip
::    ::       [end of output]
::    ::
::    ::   note: This error originates from a subprocess, and is likely not a problem with pip.
::    :: error: subprocess-exited-with-error
::    ::
::    :: × pip subprocess to install build dependencies did not run successfully.
::    :: │ exit code: 1
::    :: ╰─> See above for output.
::    ::
::    :: note: This error originates from a subprocess, and is likely not a problem with pip.
::    ::
Notice: ::    :: [notice] A new release of pip is available: 24.1.1 -> 24.3.1
Notice: ::    :: [notice] To update, run: python3 -m pip install --upgrade pip
:: Traceback (most recent call last):
::   File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 483, in <module>
::     main()
::   File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 478, in main
::     builder.build_charm()
::   File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 93, in build_charm
::     self.handle_dependencies()
::   File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 354, in handle_dependencies
::     self._install_dependencies(staging_venv_dir)
::   File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/instrum.py", line 152, in _f
::     return func(*args, **kwargs)
::   File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 284, in _install_dependencies
::     _process_run(
::   File "/snap/charmcraft/4914/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 414, in _process_run
::     raise RuntimeError(f"Subprocess command {cmd} execution failed with retcode {retcode}")
:: RuntimeError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', '--requirement=requirements.txt'] execution failed with retcode 1
:: error:
:: Traceback (most recent call last):
::   File "/snap/charmcraft/current/lib/python3.10/site-packages/craft_parts/ctl.py", line 110, in main
::     ret = CraftCtl.run(cmd, args)
::   File "/snap/charmcraft/current/lib/python3.10/site-packages/craft_parts/ctl.py", line 44, in run
::     _client(cmd, args)
::   File "/snap/charmcraft/current/lib/python3.10/site-packages/craft_parts/ctl.py", line 97, in _client
::     raise RuntimeError(message)
:: RuntimeError: Failed to run the build script for part 'charm'.
'override-build' in part 'charm' failed with code 1.
Recommended resolution: Review the scriptlet and make sure it's correct.
Failed to execute charmcraft in instance.
Full execution log: '/home/runner/.local/state/charmcraft/log/charmcraft-20241206-132340.499833.log'