Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker build ignores "build" directory to not interfere with it (+parallel build) #750

Merged
merged 2 commits into from
Jun 2, 2024

Conversation

ppalucki
Copy link
Contributor

@ppalucki ppalucki commented May 28, 2024

When developing and testing docker images at the same, I often end up with "mkdir build" error, when building docker image, this PR fixes that.

This PR also enables parallel builds inside docker (make -j) (reduction 3m-> 38s on my machine)

Error reproduction, run from root of the project:

# local building 
(mkdir build; cd build; cmake ..; cmake --build .)

# docker build (from root of the project)
docker build .

ends up with error:

 => [internal] load build definition from Dockerfile                                                                                                                                       0.0s
 => => transferring dockerfile: 503B                                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/fedora:40@sha256:4e007f288dce23966216be81ef62ba05d139b9338f327c1d1c73b7167dd47312                                                       0.0s
 => [internal] load build context                                                                                                                                                          0.6s
 => => transferring context: 40.01MB                                                                                                                                                       0.6s
 => [builder 1/4] FROM docker.io/library/fedora:40@sha256:4e007f288dce23966216be81ef62ba05d139b9338f327c1d1c73b7167dd47312                                                                 0.0s
 => CACHED [builder 2/4] RUN dnf -y install gcc-c++ git findutils make cmake                                                                                                               0.0s
 => [builder 3/4] COPY . /tmp/pcm                                                                                                                                                          0.8s
 => ERROR [builder 4/4] RUN cd /tmp/pcm && mkdir build && cd build && cmake .. && make                                                                                                     0.3s
------
 > [builder 4/4] RUN cd /tmp/pcm && mkdir build && cd build && cmake .. && make:
0.282 mkdir: cannot create directory 'build': File exists
------
Dockerfile:5
--------------------
   3 |     RUN dnf -y install gcc-c++ git findutils make cmake
   4 |     COPY . /tmp/pcm
   5 | >>> RUN cd /tmp/pcm && mkdir build && cd build && cmake .. && make
   6 |
   7 |     FROM fedora:40@sha256:4e007f288dce23966216be81ef62ba05d139b9338f327c1d1c73b7167dd47312
--------------------
ERROR: failed to solve: process "/bin/sh -c cd /tmp/pcm && mkdir build && cd build && cmake .. && make" did not complete successfully: exit code: 1

(removing outside "build" directory or using different one, solves the issue as well)

This happens becuase "build" is transfered as context to "builder" container, which tries to create "build" inside

Maybe it was intended to reause outside "build" direcotry for caching purposes, but then we're risking incoherent, no isolated builds if host environment is different from OS used in Dockerfile.

Extra gain is that with limited context, it speeds up building a little (no need to transfer unused file to container):

=> [internal] load build context                                                                                                                                                          0.6s
 => => transferring context: 40.01MB

vs with /build inside .dockerignore

 => [internal] load build context                                                                                                                                                          0.1s
 => => transferring context: 143.33kB             

dockerignore file reference: https://docs.docker.com/build/building/context/#dockerignore-files

@ppalucki ppalucki changed the title Docker build ignores "build" directory to not interfere with it Docker build ignores "build" directory to not interfere with it (+parallel build) May 28, 2024
Copy link
Contributor

@rdementi rdementi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks a lot!

@rdementi rdementi merged commit cb95e32 into intel:master Jun 2, 2024
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants