Docker build

[copyrights]

Docker build image

This PR contains a docker file for an ESPurna build environment.

Two volumes can be used.

• /espurna with ESPurna source code.
• /firmware target directory for complied firmware files.

Build

docker build -t espurna-build --build-arg UID=$(id -u) --build-arg GID=$(id -g) -f Dockerfile ..

Examples

This simple example will build all firmware files from dev branch to /tmp/firmware.

docker run --rm -it -v /tmp/firmware/:/firmware espurna-build

This example will only build firmware for environment intermittech-quinled from local files in /home/user/espurna

docker run --rm -it -v /tmp/firmware/:/firmware -v /home/user/espurna/:/espurna espurna-build intermittech-quinled

[copyrights]

I did version pinning to pass the checks, but I'm not sure if this is a good idea. It will need more maintenance with only little benefits.

[mcspr]

Pins can be relaxed imo. We don't depend on the specific version of node, but gulp. PlatformIO I would keep at 3.6.* instead though.

What would the flow be to just update internal sources? docker build --target N-step ?
Maybe just always force to use external directory? (or docker storage volume, for example)

[copyrights]

What would the flow be to just update internal sources? docker build --target N-step ?
Maybe just always force to use external directory? (or docker storage volume, for example)

Main purpose for the internal source is to install dependencies. Secondary to just build the latest dev. But maybe it would be better to use COPY instead of git clone. This should trigger the build process on changes. I will try.

[mcspr]

There is a small issue when user has different UID than 1000. We need to both remap docker-run uid and append HOME env for the npm to cache things, or it will crash when trying to write /.npm
i.e. something like this: ... -e "HOME=/tmp" -u $(id -u ${USER}):$(id -g ${USER}) ...

[mcspr]

.dockerignore is needed then (in code/ or at the top?)
we need to ignore platformio build directories
code/.pio
code/.piolibdeps
code/.pioenvs

code/node_modules

and some things from gitignore, vscode & ds_store

[mcspr]

Also, ping @wsw70 from #1739

[wsw70]

I took a different way in my automated builds (which are still in dev phase, I need to clean up the Dockerfile) - building the dev and master branches.

• the master is currently disabled because the build crashes due to ArduinoJson incompatibility (see Compile Error homeassitant.ino #1735). It is fixed in dev but not in master (@mcspr - would you know if there is a reason why it was not backported to master to avoid the build to crash?)
• I build only the firmware which is interesting for me but this is something I would like to configure - how to do it is to be assessed because there is a loop to code (over the platforms) which cannot be done directly in the Dockerfile but via some bash as part of the RUN.

My current Dockerfile is below.

FROM python:2
RUN apt update && apt install bash curl git g++
RUN pip install platformio
RUN git clone https://github.com/xoseperez/espurna.git
WORKDIR /espurna/code

# # build the master version
# RUN git checkout master
# COPY custom.h espurna/config/custom.h
# ENV PLATFORMIO_BUILD_FLAGS '-DUSE_CUSTOM_H'
# RUN pio run -e itead-sonoff-basic -e itead-sonoff-rfbridge -e itead-sonoff-touch -e aithinker-ai-light
# RUN mkdir binaries-master
# RUN cp .pioenv/itead-sonoff-basic/firmware.bin binaries-master/itead-sonoff-basic.bin
# RUN cp .pioenv/itead-sonoff-rfbridge/firmware.bin binaries-master/itead-sonoff-rfbridge.bin
# RUN cp .pioenv/itead-sonoff-touch/firmware.bin binaries-master/itead-sonoff-touch.bin
# RUN cp .pioenv/aithinker-ai-light/firmware.bin binaries-master/aithinker-ai-light.bin

# build the dev version
RUN git checkout dev
COPY custom.h espurna/config/custom.h
ENV PLATFORMIO_BUILD_FLAGS '-DUSE_CUSTOM_H'
RUN pio run -e itead-sonoff-basic -e itead-sonoff-rfbridge -e itead-sonoff-touch -e aithinker-ai-light
RUN mkdir binaries-dev && touch binaries-dev/.touch
RUN cp .pioenvs/itead-sonoff-basic/firmware.bin binaries-dev/itead-sonoff-basic.bin
RUN cp .pioenvs/itead-sonoff-rfbridge/firmware.bin binaries-dev/itead-sonoff-rfbridge.bin
RUN cp .pioenvs/itead-sonoff-touch/firmware.bin binaries-dev/itead-sonoff-touch.bin
RUN cp .pioenvs/aithinker-ai-light/firmware.bin binaries-dev/aithinker-ai-light.bin

#CMD cp -r binaries-master /binaries && cp -r binaries-dev /binaries
CMD cp -r binaries-dev /binaries

[copyrights]

There is a small issue when user has different UID than 1000. We need to both remap docker-run uid and append HOME env for the npm to cache things, or it will crash when trying to write /.npm
i.e. something like this: ... -e "HOME=/tmp" -u $(id -u ${USER}):$(id -g ${USER}) ...

I make UID and GID configurable. The home directory is inside the image on purpose so that platformio is already ready to build.
The lib_deps and npm modules are more of an issue to me. If you use your local code they get downloaded again. I thought about copying them after initialisation to a location outside the volume. Then check if e.g. .piolibdeps exists and is not empty otherwise copy them from inside the image.

[copyrights]

@wsw70 thanks for sharing your current status. I though about using FROM python:2 too, but as nodejs is needed for complete build I chose debian.

[wsw70]

@copyrights

as nodejs is needed for complete build I chose debian.

I am not really knowledgeable about platformio compilation: why is nodejs needed for a complete build? I use the platformio from python and the firmware compile fine. Is there something I am missing in my compilation?

[copyrights]

@wsw70 If you change some web files gulp is used to shrinks them to gz.h files.

[mcspr]

Which is why I wonder if there is any need to use internal source and keep libraries at all. And we also exchange small git history for docker images :/
Either that, or we need some really persistent storage (storage volume, shared between builders or current work dir)

There could be another volume exporting toolchain, located at ~/.platformio
~/.platformio/platforms/... downloads active platform. which can be different per environment
~/.platformio/packages/... keeps Core installation, xtensa toolchain, esptool
~/.platformio/.cache/... keeps library archives and network data caches (note that it might not work properly, because some of the current bugs and tz difference between container and host)

The part with COPY, pio installs and npm can be removed in favour of customizable entrypoint, either building or installing / updating dependencies.