How to install dependencies in a python sandbox using python SDK or CLI

[Madhav-Gohel]

I want to install python dependencies like pandas and numpy, manim
I tried --exec but this is the output
$ msb add app --image python --cpus 1 --memory 1024
$ msb run app --exec "pip install numpy"

Traceback (most recent call last):
  File "/usr/local/bin/pip", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/usr/local/lib/python3.13/site-packages/pip/_internal/cli/main.py", line 77, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/usr/local/lib/python3.13/site-packages/pip/_internal/commands/__init__.py", line 119, in create_command
    module = importlib.import_module(module_path)
  File "/usr/local/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1262, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1559, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1533, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1636, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1679, in _fill_cache
OSError: [Errno 24] Too many open files: '/usr/local/lib/python3.13/site-packages/pip/_internal/commands'

Also sb.command.run() (Python SDK) is not working or I am doing something wrong
Most importantly i want to use manim

[appcypher]

It is msb run app --exec pip -- install numpy.

pip install does not work with the SDK yet. I've been busy with other things so I've not had time to fix it.

[Madhav-Gohel]

This is also not working
Output:

Traceback (most recent call last):
  File "/usr/local/bin/pip", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/usr/local/lib/python3.13/site-packages/pip/_internal/cli/main.py", line 77, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/usr/local/lib/python3.13/site-packages/pip/_internal/commands/__init__.py", line 119, in create_command
    module = importlib.import_module(module_path)
  File "/usr/local/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1262, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1559, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1533, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1636, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1679, in _fill_cache
OSError: [Errno 24] Too many open files: '/usr/local/lib/python3.13/site-packages/pip/_internal/commands'

Thank you for the reply
Also, how can I add code to that specific sandbox to execute using CLI or SDK

[appcypher]

The SDK does not yet support sandboxes created within projects so you can't send code to them for execution. Sending code for execution can only be done through msb server for now.

Safe to say there are many bugs that need to be fixed and msb seems to work best on macOS rn.

[Madhav-Gohel]

While I was exploring the documentation, I found out that the sdk image can be customized https://github.com/microsandbox/microsandbox/blob/main/sdk-images/python/README.md#adding-additional-python-packages
So I customized the Dockerfile, Built it and pushed it to Docker Hub
https://hub.docker.com/r/madhavgohel/python
Then did msb pull madhavgohel/python and it returned this:

Error: http request error: error decoding response body

Caused by:
    1: error decoding response body
    2: data did not match any variant of untagged enum DockerRegistryResponse

Customized Dockerfile for reference:

# First stage: Build the portal binary with Python feature enabled
FROM rust:slim AS builder

# Install dependencies and build the portal binary
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    pkg-config \
    libssl-dev \
    && apt-get clean && rm -rf /var/lib/apt/lists/* \
    && mkdir -p /usr/src/microsandbox

WORKDIR /usr/src/microsandbox
COPY . .
RUN cargo build --release --bin portal --features python

# Second stage: Create the Python image
FROM python:latest

# Set environment variables and create user
ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=1 \
    DEBIAN_FRONTEND=noninteractive

ARG USER_NAME="python-user"
ARG USER_UID="1000"
ARG USER_GID="100"

# Install system dependencies, create user, set up directories, and install Python packages in one layer
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    curl \
    git \
    wget \
    libssl-dev \
    ca-certificates \
    && apt-get clean && rm -rf /var/lib/apt/lists/* \
    # Create non-root user and necessary directories
    && useradd -m -s /bin/bash -N -u $USER_UID $USER_NAME \
    && mkdir -p /home/$USER_NAME/work \
    && chown -R $USER_NAME:$USER_GID /home/$USER_NAME \
    && mkdir -p /etc/microsandbox/portal \
    && chown -R $USER_NAME:$USER_GID /etc/microsandbox \
    # Install core Python packages
    && pip install --no-cache-dir --upgrade pip setuptools wheel \
    && pip install --no-cache-dir \
        black \
        flake8 \
        mypy \
        pytest \
        pytest-cov \
        requests \
        ipython \
        numpy \
        pandas \
        matplotlib \
        seaborn \
        beautifulsoup4

# Copy the portal binary from the builder stage and set permissions
COPY --from=builder /usr/src/microsandbox/target/release/portal /usr/local/bin/
RUN chmod +x /usr/local/bin/portal

# Switch to the non-root user
USER $USER_NAME
WORKDIR /home/$USER_NAME/work

# Set a command that starts portal and keeps the container running
CMD ["bash", "-c", "echo 'Python environment with microsandbox-portal ready' && RUST_LOG=debug portal & tail -f /dev/null"]

[appcypher]

microsandbox expects a multi-arch build of an image even if you are only targeting just one arch.

https://github.com/microsandbox/microsandbox/blob/66f3aac71e260c0ddcb45d04c3329bfaadfe8f02/scripts/publish_sdk_images.sh#L198-L282