[FR] Add a debian-based docker image so inventree-server can connect to USB devices directly

[MechanicalMink]

Please verify that this feature request has NOT been suggested before.

• I checked and didn't find a similar feature request

Problem statement

The Alpine Linux version of the python docker image that Inventree-Server is based on is incompatible with libusb-- See the issue that I have lodged on that project here. Because of this, there is pretty much no way to get anything in the inventree-server image to connect directly to USB devices.

Suggested solution

I have found that the debian version of the python docker image is fully compatible with libusb, and better yet, brother_ql.py works perfectly on it. In my use case, if the inventree-server were moved to that image, I would be able to plug my Brother QL-600 (an exceedingly cheap label printer) directly into my pi-hosted inventree instance. Unfortunately, the existing bare metal installation instructions don't seem to piece out how to reconstruct the inventree-server instance alone.

Describe alternatives you've considered

We could also wait for libusb to address the issue their package has in Alpine Linux containers. We could also create an entirely separate container that deals with USB devices, and connect it to inventree-server over IP. Think a CUPS server-style implementation. Unfortunately I have yet to find a package that we could plug into an Alpine Linux container that will fix this.

Examples of other systems

brother_ql_web can be containerized with the following dockerfile and compose.yml on the official debian based python image and will print to my QL-600 perfectly fine.

Dockerfile:

FROM python:3.11

WORKDIR /app
COPY ./requirements.txt /app
RUN apt-get update
RUN pip install --upgrade pip
RUN pip install --no-cache-dir -r requirements.txt
RUN apt-get install -y usbutils
RUN apt-get install -y libusb-1.0-0-dev

compose.yml:

version: "3.8"
services:
  web:
    container_name: brother_ql_web
    devices:
      - /dev/usb/lp0:/dev/usb/lp0
    command: python brother_ql_web.py
    build:
      context: .
      dockerfile: Dockerfile
    volumes:
      - ./:/app
    ports:
      - "8013:8013"
    restart: unless-stopped

Place these into the folder that contains brother_ql_web.py and the rest of that project's assets, then alter the config.example.json to read as follows:

{
  "SERVER": {
    "PORT": 8013,
    "HOST": "0.0.0.0",
    "LOGLEVEL": "WARNING",
    "ADDITIONAL_FONT_FOLDER": false
  },
  "PRINTER": {
    "MODEL": "QL-570",
    "PRINTER": "file:///dev/usb/lp0"
  },
  "LABEL": {
    "DEFAULT_SIZE": "62",
    "DEFAULT_ORIENTATION": "standard",
    "DEFAULT_FONT_SIZE": 70,
    "DEFAULT_FONTS": [
      {"family": "Minion Pro",      "style": "Semibold"},
      {"family": "Linux Libertine", "style": "Regular"},
      {"family": "DejaVu Serif",    "style": "Book"}
    ]
  },
  "WEBSITE": {
    "HTML_TITLE": "Label Designer",
    "PAGE_TITLE": "Brother QL Label Designer",
    "PAGE_HEADLINE": "Design your label and print it…"
  }
}

build with:
docker compose up -d --force-recreate --build

connect to the container's IP on port 8013 and voi-la, you can print directly from the container over USB without issue.

Do you want to develop this?

• I want to develop this.

[matmair]

Alpine was chosen for the smaller and faster to build image, I do not think it makes sense to switch for a single use case.
You can build the images yourself easily with another base image with the baseimage environment variable - that way you do not need to touch anything else.

[MechanicalMink]

Alpine was chosen for the smaller and faster to build image, I do not think it makes sense to switch for a single use case. You can build the images yourself easily with another base image with the baseimage environment variable - that way you do not need to touch anything else.

Would you please either direct me to the documentation concerning using that baseimage environment variable, or write up a brief set of instructions on how to use it? That environment variable is not present nor described in the .env file that comes with this project, and it is not described in anything docker related either.

[SchrodingersGat]

There are potentially some other reasons to use a debian based docker image:

Playwright Testing

Now that we are moving to front-end testing with playwright it would be good to be able to run these tests in the default devcontainer setup. However with the current alpine image this is not possible due to incompatibility

Build Times

Our docker build times (on some architectures) are very long right now due to some library issues with alpine. A debian image should reduce this significantly:

Match Installer

Our "installer" targets debian based distributions - it would be great to have a single set (or at least a reduced common set) of required packages.

---

This is something I've been considering for a while but have had no time to investigate. If we were to move we would need to have a debian based image which does not blow out our build times or image sizes.

---

Would you please either direct me to the documentation concerning using that baseimage environment variable, or write up a brief set of instructions on how to use it?

You should just be able to follow the ./contrib/container/Dockerfile for this information.

[MechanicalMink]

For posterity: the migration to the debian image was not trivial due in a large part to the different system package managers and different package names. The initial setup with invoke is now rather unstable and performs better when you run the commands from a shell within the modified invented-server docker container. USB functionality is working, and I will circle back here in a couple hours to couple days to post the modified dockerfile and docker-compose.yml in case anyone wants to follow in my footsteps and do a better job. Note that I am a major GitHub noob, so it would probably be best if somebody else makes a PR or whatever out of what I put here. Perhaps we can chuck it all into a giant if statement in the dockerfile and build to Debian or alpine based on an environment variable?

[SchrodingersGat]

Perhaps we can chuck it all into a giant if statement in the dockerfile and build to Debian or alpine based on an environment variable?

I would not support this - crafting the docker image is a very delicate operation. If we do want to move to a debian image, I would prefer to only support that.

[matmair]

The build time is mainly caused by 1 single package building 2-3h. I tried building and referencing it twice now and failed.

Before we touch anything regarding the image I would like to hear the opinion of @wolflu05, he has proven to be very provicient in this area

[wolflu05]

Seems like only the new grpcio package provides this trouble. What is the use of this package? Can we maybe supersede this package?

Where I initially refactored to docker image to use alpine instead of debian this was not part of inventree and therefore the build time was only slightly higher, but totally fine for that huge reduction of the image size. See #5007 (comment)

Regarding the change, I do not really care, but we should keep the image size in mind. Having an image that is again > 1GB would not be optimal. Also switching would mean, users need to update their automatic deployment (e.g. when using custom system packages like the cups headers).

[wolflu05]

Just read that blog post from GitHub. Will also be interesting to speed up things in general for ARM, when GitHub releases this to open source projects by the end of this year like they stated. https://github.blog/2024-06-03-arm64-on-github-actions-powering-faster-more-efficient-build-systems/

[MechanicalMink]

I wanted to circle back to this and provide the working dockerfile and docker-compose.yml to build a debian-based usb-capable inventree-server image. I'm fairly certain I broke the dev server, since I was just bug whack-a-mole-ing and I had to import the source code in that image for some reason. As I said before, I'm a total git noob and I'd prefer somebody else takes this the rest of the way.

Dockerfile:

# The InvenTree dockerfile provides two build targets:
#
# production:
# - Required files are copied into the image
# - Runs InvenTree web server under gunicorn
#
# dev:
# - Expects source directories to be loaded as a run-time volume
# - Runs InvenTree web server under django development server
# - Monitors source files for any changes, and live-reloads server

ARG base_image=python:3.11
FROM ${base_image} AS inventree_base

# Build arguments for this image
ARG commit_tag=""
ARG commit_hash=""
ARG commit_date=""

ARG data_dir="data"

ENV PYTHONUNBUFFERED 1
ENV PIP_DISABLE_PIP_VERSION_CHECK 1
ENV INVOKE_RUN_SHELL="/bin/bash"

ENV INVENTREE_DOCKER="true"

# InvenTree paths
ENV INVENTREE_HOME="/home/inventree"
ENV INVENTREE_DATA_DIR="${INVENTREE_HOME}/${data_dir}"
ENV INVENTREE_STATIC_ROOT="${INVENTREE_DATA_DIR}/static"
ENV INVENTREE_MEDIA_ROOT="${INVENTREE_DATA_DIR}/media"
ENV INVENTREE_BACKUP_DIR="${INVENTREE_DATA_DIR}/backup"
ENV INVENTREE_PLUGIN_DIR="${INVENTREE_DATA_DIR}/plugins"

ENV INVENTREE_BACKEND_DIR="${INVENTREE_HOME}/src/backend"

# InvenTree configuration files
ENV INVENTREE_CONFIG_FILE="${INVENTREE_DATA_DIR}/config.yaml"
ENV INVENTREE_SECRET_KEY_FILE="${INVENTREE_DATA_DIR}/secret_key.txt"
ENV INVENTREE_PLUGIN_FILE="${INVENTREE_DATA_DIR}/plugins.txt"

# Worker configuration (can be altered by user)
ENV INVENTREE_GUNICORN_WORKERS="4"
ENV INVENTREE_BACKGROUND_WORKERS="4"

# Default web server address:port
ENV INVENTREE_WEB_ADDR=0.0.0.0
ENV INVENTREE_WEB_PORT=8000

LABEL org.label-schema.schema-version="1.0" \
      org.label-schema.build-date=${DATE} \
      org.label-schema.vendor="inventree" \
      org.label-schema.name="inventree/inventree" \
      org.label-schema.url="https://hub.docker.com/r/inventree/inventree" \
      org.label-schema.vcs-url="https://github.com/inventree/InvenTree.git" \
      org.label-schema.vcs-ref=${commit_tag}

# Install required system level packages
RUN apt-get update
RUN apt-get install -y \
    git gettext python3-cryptography \
    # Image format support
    libjpeg-dev libwebp-dev zlib1g-dev \
    # Weasyprint requirements : https://doc.courtbouillon.org/weasyprint/stable/first_steps.html#alpine-3-12
    python3-pip python3-pil python3-cffi python3-brotli libpango1.0-dev poppler-utils libldap2-dev \
    # Postgres client
    postgresql-client \
    # MySQL / MariaDB client
    mariadb-client libmariadb-dev-compat \
    # Simple Authentication and Security Layer
    libsasl2-dev \
    # USB packages
    usbutils libusb-1.0-0-dev \
    && \
    # fonts
    apt-get install -y fontconfig fonts-freefont-ttf fonts-noto xfonts-terminus && fc-cache -f

EXPOSE 8000

RUN mkdir -p ${INVENTREE_HOME}
WORKDIR ${INVENTREE_HOME}

COPY contrib/container/requirements.txt base_requirements.txt
COPY src/backend/requirements.txt ./
COPY contrib/container/install_build_packages.sh .
RUN chmod +x install_build_packages.sh

# For ARMv7 architecture, add the piwheels repo (for cryptography library)
# Otherwise, we have to build from source, which is difficult
# Ref: https://github.com/inventree/InvenTree/pull/4598
RUN if [ `dpkg --print-architecture` = "armhf" ]; then \
    printf "[global]\nextra-index-url=https://www.piwheels.org/simple\n" > /etc/pip.conf ; \
    fi

COPY tasks.py contrib/container/gunicorn.conf.py contrib/container/init.sh ./
RUN chmod +x init.sh

ENTRYPOINT ["/bin/bash", "init.sh"]
# use the following line for debugging
# ENTRYPOINT sleep infinity


FROM inventree_base AS prebuild
# COPY contrib/container/install_build_packages.sh .
# RUN chmod +x install_build_packages.sh
ENV PATH=/root/.local/bin:$PATH
RUN bash install_build_packages.sh
RUN pip install -r requirements.txt
RUN apt-get autoremove -y

# Frontend builder image:
FROM prebuild AS frontend

RUN apt-get update
RUN apt-get install -y nodejs npm
RUN apt-get remove cmdtest
RUN npm install -g yarn

RUN yarn config set network-timeout 600000
COPY src ${INVENTREE_HOME}/src
COPY tasks.py ${INVENTREE_HOME}/tasks.py
RUN pip install invoke
RUN cd ${INVENTREE_HOME} && inv frontend-compile

# InvenTree production image:
# - Copies required files from local directory
# - Starts a gunicorn webserver
FROM inventree_base AS production

ENV INVENTREE_DEBUG=False

# As .git directory is not available in production image, we pass the commit information via ENV
ENV INVENTREE_COMMIT_HASH="${commit_hash}"
ENV INVENTREE_COMMIT_DATE="${commit_date}"

# use dependencies and compiled wheels from the prebuild image
ENV PATH=/root/.local/bin:$PATH
COPY --from=prebuild /root/.local /root/.local


# Copy source code
# COPY src/backend/InvenTree ${INVENTREE_BACKEND_DIR}/InvenTree
# COPY src/backend/requirements.txt ${INVENTREE_BACKEND_DIR}/requirements.txt
COPY src/backend ${INVENTREE_BACKEND_DIR}/
COPY --from=frontend ${INVENTREE_BACKEND_DIR}/InvenTree/web/static/web ${INVENTREE_BACKEND_DIR}/InvenTree/web/static/web

# Launch the production server
CMD gunicorn -c ./gunicorn.conf.py InvenTree.wsgi -b 0.0.0.0:8000 --chdir ${INVENTREE_BACKEND_DIR}/InvenTree

FROM inventree_base AS dev

# Vite server (for local frontend development)
EXPOSE 5173

# Install packages required for building python packages
# RUN chmod +x install_build_packages.sh
# RUN ls && install_build_packages.sh

RUN pip install --require-hashes -r base_requirements.txt --no-cache

# Install nodejs / npm / yarn

RUN apt-get update
RUN apt-get install -y nodejs npm
RUN apt-get remove cmdtest
RUN npm install -g yarn
RUN yarn config set network-timeout 600000

# The development image requires the source code to be mounted to /home/inventree/
# So from here, we don't actually "do" anything, apart from some file management
# jacob note: for some reason, it seems like we're falling straight through to the dev server area.
# I will try to copy server code over here...
COPY src/backend/InvenTree ${INVENTREE_BACKEND_DIR}/InvenTree
COPY src/backend/requirements.txt ${INVENTREE_BACKEND_DIR}/requirements.txt
# RUN ls && sleep 10000
COPY --from=frontend ${INVENTREE_BACKEND_DIR}/InvenTree/web/static/web ${INVENTREE_BACKEND_DIR}/InvenTree/web/static/web


ENV INVENTREE_DEBUG=True

# Location for python virtual environment
# If the INVENTREE_PY_ENV variable is set, the entrypoint script will use it!
ENV INVENTREE_PY_ENV="${INVENTREE_DATA_DIR}/env"

WORKDIR ${INVENTREE_HOME}

# Entrypoint ensures that we are running in the python virtual environment
ENTRYPOINT ["/bin/bash", "init.sh"]
# use the following line for debugging
# ENTRYPOINT sleep infinity


# Launch the development server
CMD ["invoke", "server", "-a", "${INVENTREE_WEB_ADDR}:${INVENTREE_WEB_PORT}"]

docker-compose.yml:

version: "3.8"

# Docker compose recipe for a production-ready InvenTree setup, with the following containers:
# - PostgreSQL as the database backend
# - gunicorn as the InvenTree web server
# - django-q as the InvenTree background worker process
# - Caddy as a reverse proxy
# - redis as the cache manager (optional, disabled by default)

# ---------------------
# READ BEFORE STARTING!
# ---------------------

# -----------------------------
# Setting environment variables
# -----------------------------
# Shared environment variables should be stored in the .env file
# Changes made to this file are reflected across all containers!
#
# IMPORTANT NOTE:
# You should not have to change *anything* within this docker-compose.yml file!
# Instead, make any changes in the .env file!

# ------------------------
# InvenTree Image Versions
# ------------------------
# By default, this docker-compose script targets the STABLE version of InvenTree,
# image: inventree/inventree:stable
#
# To run the LATEST (development) version of InvenTree,
# change the INVENTREE_TAG variable (in the .env file) to "latest"
#
# Alternatively, you could target a specific tagged release version with (for example):
# INVENTREE_TAG=0.7.5
#

services:
    # Database service
    # Use PostgreSQL as the database backend
    inventree-db:
        image: postgres:13
        container_name: inventree-db
        expose:
            - ${INVENTREE_DB_PORT:-5432}/tcp
        environment:
            - PGDATA=/var/lib/postgresql/data/pgdb
            - POSTGRES_USER=${INVENTREE_DB_USER:?You must provide the 'INVENTREE_DB_USER' variable in the .env file}
            - POSTGRES_PASSWORD=${INVENTREE_DB_PASSWORD:?You must provide the 'INVENTREE_DB_PASSWORD' variable in the .env file}
            - POSTGRES_DB=${INVENTREE_DB_NAME:?You must provide the 'INVENTREE_DB_NAME' variable in the .env file}
        volumes:
            # Map 'data' volume such that postgres database is stored externally
            - ${INVENTREE_EXT_VOLUME:?You must specify the 'INVENTREE_EXT_VOLUME' variable in the .env file!}:/var/lib/postgresql/data/:z
        restart: unless-stopped

    # redis acts as database cache manager
    # only runs under the "redis" profile : https://docs.docker.com/compose/profiles/
    inventree-cache:
        image: redis:7.0
        container_name: inventree-cache
        depends_on:
            - inventree-db
        profiles:
            - redis
        env_file:
            - .env
        expose:
            - ${INVENTREE_CACHE_PORT:-6379}
        restart: always

    # InvenTree web server service
    # Uses gunicorn as the web server
    inventree-server:
        # If you wish to specify a particular InvenTree version, do so here
        # image: inventree/inventree:${INVENTREE_TAG:-stable}
        build:
          context: ../../
          dockerfile: ./contrib/container/Dockerfile
        container_name: inventree-server
        devices:
            - /dev/usb/lp0:/dev/usb/lp0
        #Only use the above line if you actually have a device plugged in during container bringup, otherwise comment it out
        # Only change this port if you understand the stack.
        expose:
            - 8000
        depends_on:
            - inventree-db
        env_file:
            - .env
        volumes:
            # Data volume must map to /home/inventree/data
            - ${INVENTREE_EXT_VOLUME}:/home/inventree/data:z
        restart: unless-stopped

    # Background worker process handles long-running or periodic tasks
    inventree-worker:
        # If you wish to specify a particular InvenTree version, do so here
        image: inventree/inventree:${INVENTREE_TAG:-stable}
        container_name: inventree-worker
        command: invoke worker
        depends_on:
            - inventree-server
        env_file:
            - .env
        volumes:
            # Data volume must map to /home/inventree/data
            - ${INVENTREE_EXT_VOLUME}:/home/inventree/data:z
        restart: unless-stopped

    # caddy acts as reverse proxy and static file server
    # https://hub.docker.com/_/caddy
    inventree-proxy:
        container_name: inventree-proxy
        image: caddy:alpine
        restart: always
        depends_on:
            - inventree-server
        ports:
            - ${INVENTREE_WEB_PORT:-80}:80
            - 443:443
        env_file:
            - .env
        volumes:
            - ./Caddyfile:/etc/caddy/Caddyfile:ro,z
            - ${INVENTREE_EXT_VOLUME}/static:/var/www/static:z
            - ${INVENTREE_EXT_VOLUME}/media:/var/www/media:z
            - ${INVENTREE_EXT_VOLUME}:/var/log:z
            - ${INVENTREE_EXT_VOLUME}:/data:z
            - ${INVENTREE_EXT_VOLUME}:/config:z

    # alternative: run nginx as reverse proxy
    # inventree-proxy:
    #     container_name: inventree-proxy
    #     image: nginx:stable
    #     restart: always
    #     depends_on:
    #         - inventree-server
    #     ports:
    #         - ${INVENTREE_WEB_PORT:-80}:80
    #         - 443:443
    #     volumes:
    #         - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro,z
    #         - ${INVENTREE_EXT_VOLUME}:/var/www:z

I haven't changed much, but it took a long-ass time to get there. This definitely builds on an 8gb raspberry pi 4 running the current Pi OS. Make sure you also have at least 8gb free on the filesystem to build-- Again, I'm a noob and I doubt what I did was the most efficient. Plz improve.

Additionally, if you want to use this for my purposes-- printing labels-- remember to downgrade the Inventree venv's version of PIL to 9.5.0, unless you figure out a way to add that into the dockerfile.

Best of luck, I have no idea what I'm doing. I'll link to the issue wherein I provide code to run Brother QL-570s and QL-600s via USB directly from the container momentarily.

All that said, having had to write a printer driver on top of all of this anyway, I'm beginning to wonder if the smart solution is to leave the inventree-server image alone and create a separate image altogether that builds from debian and provides USB printing plugins. We could also provide that end of the socket as a cups-style daemon to run directly on the host system. Basically what I'm saying is I'd like to follow the progress of development but I'm tired of waiting 2 hours for inventree-server to compile, so for now I've got what I've got.

[SchrodingersGat]

What is the use of this package? Can we maybe supersede this package?

@wolflu05 we need this for tracing support - it would be hard to work around this and still support the feature

[wolflu05]

I'm beginning to wonder if the smart solution is to leave the inventree-server image alone and create a separate image altogether that builds from debian and provides USB printing plugins.

That's what I was thinking a while ago too for my cups plugin, but I don't have that spare time to implement this right now. I thought about building a custom image for my cups plugin that just exposes the required native apis via a REST or socket interface which then runs in a second container where the printing driver running on the inventree worker communicates with via an internal network.

[github-actions]

This issue seems stale. Please react to show this is still important.