errors in mamba (is it relocatable?) #413

ddobrinskiy · 2023-02-19T12:55:38Z

Getting an ssl error when using mamba installed from tea

pkg identifier: github.com/mamba-org/mamba

Steps to reproduce:

❯ sh <(curl tea.xyz) mamba init "$(basename "${SHELL}")"
❯ source ~/.zshrc
❯ whereis mamba
mamba: /home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/bin/mamba
❯ mamba install requests --yes
    RuntimeError: Download error (77) Problem with the SSL CA cert (path? access rights?) [https://conda.anaconda.org/conda-forge/noarch/repodata.json]
    error setting certificate file: /opt/github.com/mamba-org/mamba/v22.11.1.4/ssl/cacert.pem

See detailed error log


$ mamba install requests --yes
Download error (77) Problem with the SSL CA cert (path? access rights?) [https://conda.anaconda.org/conda-forge/noarch/repodata.json]

error setting certificate file: /opt/github.com/mamba-org/mamba/v22.11.1.4/ssl/cacert.pem
>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
  File "/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/lib/python3.10/site-packages/conda/exceptions.py", line 1118, in __call__
    return func(*args, **kwargs)
  File "/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/lib/python3.10/site-packages/mamba/mamba.py", line 936, in exception_converter
    raise e
  File "/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/lib/python3.10/site-packages/mamba/mamba.py", line 929, in exception_converter
    exit_code = _wrapped_main(*args, **kwargs)
  File "/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/lib/python3.10/site-packages/mamba/mamba.py", line 887, in _wrapped_main
    result = do_call(parsed_args, p)
  File "/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/lib/python3.10/site-packages/mamba/mamba.py", line 750, in do_call
    exit_code = install(args, parser, "install")
  File "/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/lib/python3.10/site-packages/mamba/mamba.py", line 497, in install
    index = load_channels(pool, channels, repos)
  File "/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/lib/python3.10/site-packages/mamba/utils.py", line 129, in load_channels
    index = get_index(
  File "/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/lib/python3.10/site-packages/mamba/utils.py", line 110, in get_index
    is_downloaded = dlist.download(api.MAMBA_DOWNLOAD_FAILFAST)
RuntimeError: Download error (77) Problem with the SSL CA cert (path? access rights?) [https://conda.anaconda.org/conda-forge/noarch/repodata.json]
error setting certificate file: /opt/github.com/mamba-org/mamba/v22.11.1.4/ssl/cacert.pem

$ /home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/bin/mamba install jupyterlab --yes
environment variables:

CIO_TEST=

CONDA_DEFAULT_ENV=base

CONDA_EXE=/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/bin/conda

CONDA_PREFIX=/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4

CONDA_PREFIX_1=/home/david/proj/pkgdev/pantry.extra/tea.out/github.com/mamba-

org/mamba/v22.11.1.4

CONDA_PROMPT_MODIFIER=(base)

CONDA_PYTHON_EXE=/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/bin/python

CONDA_ROOT=/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4

CONDA_SHLVL=2

CURL_CA_BUNDLE=

LD_PRELOAD=

PATH=/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/bin:/home/david

/proj/pkgdev/pantry.extra/tea.out/github.com/mamba-org/mamba/v22.11.1.

4/condabin:/home/david/.tea/tea.xyz/v0.24.3/bin:/home/david/.local/bin

:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/gam

es:/usr/local/games:/snap/bin:/snap/bin:/home/david/.local/bin:/home/d

avid/.oh-my-zsh/custom/plugins/fzf-zsh-

plugin/bin:/home/david/.fzf/bin:/home/david/.tea/ruby-

lang.org/v3/bin:/home/david/.tea/crates.io/fd-find/v*/bin:/home/david/

.tea/crates.io/ripgrep/v*/bin:/home/david/.tea/github.com/junegunn/fzf

/v*/bin:/home/david/.tea/crates.io/bat/v*/bin:/home/david/.cargo/bin:/

home/david/.local/bin:/home/david/.local/bin:/home/david/.oh-my-

zsh/custom/plugins/fzf-zsh-plugin/bin:/home/david/.tea/crates.io/bat/v

/bin:/home/david/.tea/crates.io/fd-find/v/bin:/home/david/.tea/crate

s.io/ripgrep/v*/bin:/home/david/.tea/github.com/charliermarsh/ruff/v*/

bin:/home/david/.tea/github.com/junegunn/fzf/v*/bin:/home/david/.tea/r

uby-lang.org/v*/bin:/home/david/.cargo/bin:/home/david/.local/bin:/hom

e/david/.oh-my-zsh/custom/plugins/fzf-zsh-plugin/bin:/home/david/.tea/

crates.io/bat/v*/bin:/home/david/.tea/crates.io/fd-find/v*/bin:/home/d

avid/.tea/crates.io/ripgrep/v*/bin:/home/david/.tea/github.com/charlie

rmarsh/ruff/v*/bin:/home/david/.tea/github.com/junegunn/fzf/v*/bin:/ho

me/david/.tea/ruby-

lang.org/v*/bin:/home/david/.cargo/bin:/home/david/.local/bin

REQUESTS_CA_BUNDLE=

SSL_CERT_FILE=

TMUX_PLUGIN_MANAGER_PATH=/home/david/.tmux/plugins

WINDOWPATH=2
 active environment : base
active env location : /home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4
        shell level : 2
   user config file : /home/david/.condarc

populated config files : /home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/.condarc

conda version : 22.11.1

conda-build version : not installed

python version : 3.10.9.final.0

virtual packages : __archspec=1=x86_64

__glibc=2.36=0

__linux=5.19.0=0

__unix=0=0

base environment : /home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4  (writable)

conda av data dir : /home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/etc/conda

conda av metadata url : None

channel URLs : https://conda.anaconda.org/conda-forge/linux-64

https://conda.anaconda.org/conda-forge/noarch

package cache : /home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/pkgs

/home/david/.conda/pkgs

envs directories : /home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/envs

/home/david/.conda/envs

platform : linux-64

user-agent : conda/22.11.1 requests/2.28.2 CPython/3.10.9 Linux/5.19.0-31-generic ubuntu/22.10 glibc/2.36

UID:GID : 1000:1000

netrc file : None

offline mode : False
An unexpected error has occurred. Conda has prepared the above report.

Looking for: ['jupyterlab']

The text was updated successfully, but these errors were encountered:

ddobrinskiy · 2023-02-19T13:05:38Z

Will look deeper into it tomorrow

mxcl · 2023-02-19T13:37:11Z

add curl.se/ca-certs to dependencies?

ddobrinskiy · 2023-02-20T09:49:00Z

Huh, I have a feeling this will be a tricky one.

This error does not reproduce if I build mamba locally:

❯ pwd
/home/david/proj/pkgdev/pantry.extra
❯ tea build github.com/mamba-org/mamba # works ok
❯ tea test github.com/mamba-org/mamba   # works ok

The original error, when installing from tea natively, contains the following error (notice the path to cacert.pem):

RuntimeError: Download error (77) Problem with the SSL CA cert (path? access rights?) [https://conda.anaconda.org/conda-forge/noarch/repodata.json]
error setting certificate file: /opt/github.com/mamba-org/mamba/v22.11.1.4/ssl/cacert.pem

AFAIK, /opt/ is the default build location of the github-actions runner.

When looking into the logs, I noticed that mamba does some linking at build-time

Transaction starting 
...
Linking ca-certificates-2022.12.7-ha878542_0
...

To sum it up, mamba probably hard-codes the ssl cert location at build time, will look into where it's stored and how to make the link universal, like what we do with fix-shebangs.ts

ddobrinskiy · 2023-02-20T10:06:42Z

The good news is that the new test will catch this behavior, because github actions helpfully use different paths at build and at test stages

https://github.com/teaxyz/pantry.extra/actions/runs/4222137543/jobs/7330371918

ddobrinskiy · 2023-02-20T10:09:08Z

Found the main offender

❯ grep '/opt/.*cacert.pem' ~/.tea/github.com/mamba-org/mamba

/home/david/.tea/github.com/mamba-org/mamba/v22.11.1.4/bin/curl-config
77:        echo "/opt/github.com/mamba-org/mamba/v22.11.1.4/ssl/cacert.pem"
185:        echo " '--prefix=/opt/github.com/mamba-org/mamba/v22.11.1.4' '--host=x86_64-conda-linux-gnu' '--disable-ldap' '--with-ca-bundle=/opt/github.com/mamba-org/mamba/v22.11.1.4/ssl/cacert.pem' '--with-openssl=/opt/github.com/mamba-org/mamba/v22.11.1.4' '--with-zlib=/opt/github.com/mamba-org/mamba/v22.11.1.4' '--with-gssapi=/opt/github.com/mamba-org/mamba/v22.11.1.4' '--with-libssh2=/opt/github.com/mamba-org/mamba/v22.11.1.4' '--with-nghttp2=/opt/github.com/mamba-org/mamba/v22.11.1.4' 'build_alias=x86_64-conda-linux-gnu' 'host_alias=x86_64-conda-linux-gnu' 'CC=/home/conda/feedstock_root/build_artifacts/curl_split_recipe_1671621346713/_build_env/bin/x86_64-conda-linux-gnu-cc' 'CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/github.com/mamba-org/mamba/v22.11.1.4/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/curl_split_recipe_1671621346713/work=/usr/local/src/conda/curl_split_recipe-7.87.0 -fdebug-prefix-map=/opt/github.com/mamba-org/mamba/v22.11.1.4=/usr/local/src/conda-prefix -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /opt/github.com/mamba-org/mamba/v22.11.1.4/include' 'LDFLAGS=-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/opt/github.com/mamba-org/mamba/v22.11.1.4/lib -Wl,-rpath-link,/opt/github.com/mamba-org/mamba/v22.11.1.4/lib -L/opt/github.com/mamba-org/mamba/v22.11.1.4/lib' 'CPPFLAGS=-DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /opt/github.com/mamba-org/mamba/v22.11.1.4/include' 'CPP=/home/conda/feedstock_root/build_artifacts/curl_split_recipe_1671621346713/_build_env/bin/x86_64-conda-linux-gnu-cpp'"

ddobrinskiy · 2023-02-20T10:12:14Z

But the issue is more widespread: this command get over 800 hits

❯ grep '/opt/github.com/mamba-org/mamba' ~/.tea/github.com/mamba-org/mamba

opt_offenders.txt

There is probably an easy fix with some env variable here? Will try to look into it

ddobrinskiy · 2023-02-20T10:45:05Z

Basically we want every hard-coded piece of /opt/github.com/mamba-org/ generated at build converted to $TEA_PREFIX/github.com/mamba-org/

But if we look into opt_offenders.txt, there is a lot of stuff to rename, and some of it is hardcoded - meaning we can't just replace it with a link to env var

I feel like we're getting into build/rpath territory here, where I haven't gone before.

@mxcl any easy fixes that you may be aware of?

ddobrinskiy · 2023-02-20T11:04:18Z

I suggest we remove mamba from pantry until this is resolved, see #415

ddobrinskiy · 2023-02-20T13:29:48Z

This seems to be subset of a larger problem, i.e. mamba/conda is not relocatable after building it:

https://docs.anaconda.com/anaconda/user-guide/tasks/move-directory/

If you simply copy the Anaconda files to a new directory, Anaconda will not work.
To move Anaconda from one directory to another:

Uninstall Anaconda.

Go to the new directory and install it there following the Anaconda installation instructions.

A possible solution is to distribute the installer via tea, and force it to build locally when a user tries to install? Not sure how feasible this is, but it would surely solve a lot of problems.

Because currently at build mamba creates 800+ files with the prefix location hard-coded

ddobrinskiy · 2023-02-20T17:45:01Z

Extra error, for posterity: installing extra packages into an environment other than default fails:

❯ mamba activate myenv
❯ mamba install some_package --yes
Traceback (most recent call last):
  File "/home/david/proj/pkgdev/pantry.extra/tea.out/github.com/mamba-org/mamba/v22.11.1.4/bin/mamba", line 7, in <module>
    from mamba.mamba import main
ModuleNotFoundError: No module named 'mamba'

temporary fix: install from default env
mamba install -n myenv some_package --yes

mxcl · 2023-02-24T13:30:43Z

We can make it relocatable. It just requires work.

ddobrinskiy linked a pull request Feb 20, 2023 that will close this issue

fix(mamba) #414

Open

ddobrinskiy mentioned this issue Feb 20, 2023

remove mamba (until fixed) #415

Closed

ddobrinskiy changed the title ~~SSL error: mamba~~ errors in mamba (is it relocatable?) Feb 20, 2023

mxcl added the bug Something isn't working label Mar 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

errors in mamba (is it relocatable?) #413

errors in mamba (is it relocatable?) #413

ddobrinskiy commented Feb 19, 2023

>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

ddobrinskiy commented Feb 19, 2023

mxcl commented Feb 19, 2023

ddobrinskiy commented Feb 20, 2023 •

edited

Loading

ddobrinskiy commented Feb 20, 2023 •

edited

Loading

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

mxcl commented Feb 24, 2023

errors in mamba (is it relocatable?) #413

errors in mamba (is it relocatable?) #413

Comments

ddobrinskiy commented Feb 19, 2023

>>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

ddobrinskiy commented Feb 19, 2023

mxcl commented Feb 19, 2023

ddobrinskiy commented Feb 20, 2023 • edited Loading

ddobrinskiy commented Feb 20, 2023 • edited Loading

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

ddobrinskiy commented Feb 20, 2023

mxcl commented Feb 24, 2023

ddobrinskiy commented Feb 20, 2023 •

edited

Loading

ddobrinskiy commented Feb 20, 2023 •

edited

Loading