-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify registry build #20622
base: main
Are you sure you want to change the base?
Simplify registry build #20622
Conversation
1. `make/photon/Makefile` Remove `cd` command since the current directory is changed in `make/photon/registry/builder`. 2. `make/photon/registry/Dockerfile.base` Change the owner of `/etc/pki/tls/certs` when building the base image. (Since files under `/etc/pki/tls/certs` are not affected by binary build.) 3. `make/photon/registry/Dockerfile` - Remove changing the owner of `/etc/pki/tls/certs` since the change has been moved to the base image. - The owner/permission changes of files copied from the context are now performed simultaneously when the `COPY` command is executed. (If `COPY` and the owner/permission changes were separated, both image layers before and after change would be created, making the image unnecessarily large.) - Add `--link` option to `COPY` command. (This will improve image build efficiency since the base image will not be extracted at build time.) 4. `make/photon/registry/builder`. - Move `set -e` (exit immediately on error) to the top. - There is no `error` command, so change it to the `echo` command. - Remove `cur` variables that are no longer used by using `~-` and `cd -`. - Add `--depth 1` option to `git clone` command. (Since we only need the specified version of the source to build, we don't need the whole history, and this reduces the amount of transfer at clone time.) - Change the `docker build` command to specify the source file directly with the `-f` option instead of copying `Dockerfile.binary`. - Change the `docker build` command to directly output the binary file without creating a container image and container by specifying the output directory with the `-o` option. 5. `make/photon/registry/Dockerfile.binary`. - Remove `PREFIX` variable specified at `make` command line, since it is not used. - Change `make` build target from `binaries` to `bin/registry`. (Since `binaries` other than `bin/registry` are not used.) - Add a stage to extract only binary files, since `make/photon/registry/builder` now outputs binary files directly. Signed-off-by: Mitsuru Kariya <mitsuru.kariya@nttdata.com>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #20622 +/- ##
===========================================
+ Coverage 45.36% 66.73% +21.36%
===========================================
Files 244 1047 +803
Lines 13333 114271 +100938
Branches 2719 2847 +128
===========================================
+ Hits 6049 76259 +70210
- Misses 6983 34132 +27149
- Partials 301 3880 +3579
Flags with carried forward coverage won't be shown. Click here to find out more. |
make/photon/registry/builder
Outdated
|
||
# the temp folder to store distribution source code... | ||
TEMP=`mktemp -d ${TMPDIR-/tmp}/distribution.XXXXXX` | ||
git clone -b $VERSION https://github.com/distribution/distribution.git $TEMP | ||
git clone -b $VERSION --depth 1 https://github.com/distribution/distribution.git $TEMP |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is it necessary to specify the depth here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If --depth 1
is not specified, then the entire history of all the commits to the $VERSION
will be cloned, which is unnecessary and is wasteful in terms of transfer amount and processing time (and very painful for a poor environment like mine).
By specifying --depth 1
, only the commits specified by $VERSION
can be retrieved.
|
||
# add patch redis | ||
cd $TEMP | ||
git apply $cur/redis.patch | ||
cd $cur | ||
git apply ~-/redis.patch |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does it support all the linux OS versions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes.
It is supported by all currently used bash.
RUN CGO_ENABLED=0 make clean bin/registry | ||
|
||
FROM scratch | ||
COPY --from=build /go/src/github.com/docker/distribution/bin/registry / |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
to /bin/binary?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was originally created in binary
, not bin/binary
, does that mean it should be changed to bin/binary
?
@@ -7,4 +7,7 @@ ENV GO111MODULE auto | |||
WORKDIR $DISTRIBUTION_DIR | |||
COPY . $DISTRIBUTION_DIR | |||
|
|||
RUN CGO_ENABLED=0 make PREFIX=/go clean binaries | |||
RUN CGO_ENABLED=0 make clean bin/registry | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this change needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PREFIX
variable is not used anywhere, which leads to confusion.
Therefore, I think it is better to remove it.
make/photon/Makefile
Outdated
@@ -178,7 +178,7 @@ _build_registry: | |||
rm -rf $(DOCKERFILEPATH_REG)/binary && mkdir -p $(DOCKERFILEPATH_REG)/binary && \ | |||
$(call _get_binary, $(REGISTRYURL), $(DOCKERFILEPATH_REG)/binary/registry); \ | |||
else \ | |||
cd $(DOCKERFILEPATH_REG) && $(DOCKERFILEPATH_REG)/builder $(REGISTRY_SRC_TAG) && cd - ; \ | |||
$(DOCKERFILEPATH_REG)/builder $(REGISTRY_SRC_TAG) ; \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does this mean the builder script cannot be executed in the location of $(DOCKERFILEPATH_REG)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No.
The builder script can be executed in the location of $(DOCKERFILEPATH_REG)
, but it also can be executed in any other location, so it need not to cd to $(DOCKERFILEPATH_REG)
.
Signed-off-by: Mitsuru Kariya <mitsuru.kariya@nttdata.com>
Signed-off-by: Mitsuru Kariya <mitsuru.kariya@nttdata.com>
Modify `FROM as` to `FROM AS` to suppress Docker warnings Signed-off-by: Mitsuru Kariya <mitsuru.kariya@nttdata.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Take note that this modification relies on certain new features of Docker Buildkit. As a result, you might encounter an error when building with an older version of Docker. |
Do you have an idea how new the docker version should be? I scanned the Dockerfile, and based on the output it should run on Docker 17.06 CE (June 2017) We should be pritty save on that side |
AFAIK, |
Comprehensive Summary of your change
make/photon/Makefile
Remove
cd
command since the current directory is changed inmake/photon/registry/builder
.make/photon/registry/Dockerfile.base
Change the owner of
/etc/pki/tls/certs
when building the base image.(Since files under
/etc/pki/tls/certs
are not affected by binary build.)make/photon/registry/Dockerfile
/etc/pki/tls/certs
since the change has been moved to the base image.COPY
command is executed.(If
COPY
and the owner/permission changes were separated, both image layers before and after change would be created, making the image unnecessarily large.)--link
option toCOPY
command.(This will improve image build efficiency since the base image will not be extracted at build time.)
make/photon/registry/builder
.set -e
(exit immediately on error) to the top.error
command, so change it to theecho
command.cur
variables that are no longer used by using~-
andcd -
.--depth 1
option togit clone
command.(Since we only need the specified version of the source to build, we don't need the whole history, and this reduces the amount of transfer at clone time.)
docker build
command to specify the source file directly with the-f
option instead of copyingDockerfile.binary
.docker build
command to directly output the binary file without creating a container image and container by specifying the output directory with the-o
option.make/photon/registry/Dockerfile.binary
.PREFIX
variable specified atmake
command line, since it is not used.make
build target frombinaries
tobin/registry
.(Since
binaries
other thanbin/registry
are not used.)make/photon/registry/builder
now outputs binary files directly.Issue being fixed
Please indicate you've done the following: