-
Notifications
You must be signed in to change notification settings - Fork 548
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI] Optimize rosetta docker #14272
Merged
Merged
[CI] Optimize rosetta docker #14272
Changes from 29 commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
eab5933
New rosetta docker
dkijania 498533f
comp fix
dkijania 9459d39
fix from
dkijania 435d430
fix rosetta docker
dkijania 71cf695
better deb
dkijania 12b4aaf
Merge branch 'berkeley' into clean_rosetta_toolchain_dockers
dkijania 20cd9f7
clean deb job and dependency
dkijania d90fcf1
Merge remote-tracking branch 'origin/clean_rosetta_toolchain_dockers'…
dkijania f089566
Merge branch 'berkeley' into clean_rosetta_toolchain_dockers
dkijania 1ba559b
fix cp
dkijania ecf8121
create dirs
dkijania d9ec4f0
set variable
dkijania dc42234
fix network
dkijania 3547647
join rosetta with mina
dkijania 47738a0
do not download mina-rosetta
dkijania 1907c7e
remove copy
dkijania ec49b33
convert mina-dev mina
dkijania c731989
fix ocaml signer
dkijania 95b5a7c
fix path from /rosetta to /etc/mina/rosetta/
dkijania a1ebbf9
Merge branch 'berkeley' into clean_rosetta_toolchain_dockers
dkijania 707a6c5
fix archive path
dkijania 0cf77a3
Merge remote-tracking branch 'origin/clean_rosetta_toolchain_dockers'…
dkijania f99531d
Merge branch 'berkeley' into clean_rosetta_toolchain_dockers
dkijania c894d72
fix mina-rosetta name
dkijania ceabc21
mina archive fix
dkijania a968932
build rosetta instead of downloading
dkijania d14db90
install gcc and go
dkijania 30cd04a
Merge branch 'berkeley' into clean_rosetta_toolchain_dockers
dkijania 1cd66ba
Merge branch 'berkeley' into clean_rosetta_toolchain_dockers
dkijania 457ad03
remove comments
dkijania 1747ef2
Merge branch 'berkeley' into clean_rosetta_toolchain_dockers
dkijania f58e8c4
Merge branch 'berkeley' into clean_rosetta_toolchain_dockers
dkijania File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,8 +6,14 @@ | |
ARG image=ubuntu:focal | ||
FROM ${image} AS production | ||
ARG deb_codename=focal | ||
ARG deb_version | ||
ARG deb_release=unstable | ||
ARG network=mainnet | ||
|
||
|
||
ARG psql_version=13 | ||
# Golang version number used to detemine tarball name | ||
ARG GO_VERSION=1.19.11 | ||
|
||
ARG MINA_DAEMON_PORT=10101 | ||
ARG MINA_CONFIG_DIR=/data/.mina-config | ||
|
@@ -18,6 +24,23 @@ ARG ROSETTA_CLI_VERSION=v0.10.1 | |
|
||
ENV DEBIAN_FRONTEND noninteractive | ||
|
||
# We do not install the below platform-specific dependencies, | ||
# and instead assume that apt will install the proper deps based on the package definition. | ||
# The packages are noted here just for clarity/documentation. | ||
# Stretch-only Deps: | ||
# libffi6 | ||
# libprocps6 | ||
# libjemalloc1 | ||
# Buster-only Deps: | ||
# libffi7 | ||
# libprocps7 | ||
# libjemalloc2 | ||
# Bullseye/Focal-only Deps: | ||
# libffi7 | ||
# libprocps8 | ||
# libjemalloc2 | ||
|
||
|
||
# --- Dependencies across many platforms | ||
RUN apt-get update --quiet --yes \ | ||
&& apt-get upgrade --quiet --yes \ | ||
|
@@ -26,6 +49,7 @@ RUN apt-get update --quiet --yes \ | |
apt-transport-https \ | ||
curl \ | ||
ca-certificates \ | ||
build-essential \ | ||
dnsutils \ | ||
dumb-init \ | ||
gettext \ | ||
|
@@ -46,41 +70,16 @@ RUN curl -s https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \ | |
postgresql-"$psql_version" \ | ||
postgresql-client-"$psql_version" | ||
|
||
# --- Debian 11 "Bullseye" Dependencies | ||
RUN test "$deb_codename" = "bullseye" \ | ||
&& apt-get update -y \ | ||
&& apt-get install -y \ | ||
libjemalloc2 \ | ||
libprocps8 \ | ||
libffi7 \ | ||
|| exit 0 | ||
|
||
# --- Debian 10 "Buster" Dependencies | ||
RUN test "$deb_codename" = "buster" \ | ||
&& apt-get update --quiet --yes \ | ||
&& apt-get install --quiet --yes --no-install-recommends \ | ||
libjemalloc2 \ | ||
libprocps7 \ | ||
libffi6 \ | ||
|| exit 0 | ||
|
||
# --- Ubuntu 20.04 "Focal Fossa" Dependencies | ||
RUN test "$deb_codename" = "focal" \ | ||
&& apt-get update --quiet --yes \ | ||
&& apt-get install --quiet --yes --no-install-recommends \ | ||
libjemalloc2 \ | ||
libprocps8 \ | ||
libffi7 \ | ||
|| exit 0 | ||
|
||
# --- Install rosetta-cli | ||
|
||
|
||
# --- Golang install of a given GO_VERSION (add -v for spam output of each file from the go dist) | ||
RUN curl -s "https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz" | tar -xz -C /usr/lib/ | ||
|
||
# The following commands install rosetta-cli with an updated rosetta-sdk-go dependency, which supports delegation transactions. | ||
# Upstream PR to rosetta-sdk-go: https://github.com/coinbase/rosetta-sdk-go/pull/457 | ||
# They can be replaced once there is a new release of rosetta-cli containing the above change by: | ||
# RUN curl -sSfL https://raw.githubusercontent.com/coinbase/rosetta-cli/${ROSETTA_CLI_VERSION}/scripts/install.sh | sh -s | ||
COPY --from=builder /usr/lib/go /usr/lib/go | ||
RUN apt-get install -y gcc | ||
#RUN curl -sSfL https://raw.githubusercontent.com/coinbase/rosetta-cli/${ROSETTA_CLI_VERSION}/scripts/install.sh | sh -s | ||
RUN export GOBIN="$(pwd)/bin" \ | ||
&& curl -L "https://github.com/coinbase/rosetta-cli/archive/refs/tags/v0.10.1.tar.gz" -o "/tmp/v0.10.1.tar.gz" \ | ||
&& tar xzf "/tmp/v0.10.1.tar.gz" -C "/tmp" \ | ||
|
@@ -89,8 +88,17 @@ RUN export GOBIN="$(pwd)/bin" \ | |
&& /usr/lib/go/bin/go mod tidy \ | ||
&& /usr/lib/go/bin/go install | ||
|
||
# Rosetta-demo artifacts / build dir removal: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this documentation for running the test locally? It would be better to move this to somewhere more discoverable (or just delete it). |
||
# Make rosetta-crucial components and the generate_keypair tool | ||
# && dune build --profile=${DUNE_PROFILE} \ | ||
# && _build/default/src/app/runtime_genesis_ledger/runtime_genesis_ledger.exe \ | ||
# --config-file src/app/rosetta/demo-config.json \ | ||
# --genesis-dir ${HOME}/demo-genesis \ | ||
# && mv _build/default/src/app $HOME/app \ | ||
# && rm -rf _build | ||
|
||
# --- Generate en_US.UTF-8 locale to allow use of O(1) Labs DB dumps | ||
RUN locale-gen en_US.UTF-8 | ||
#RUN locale-gen en_US.UTF-8 | ||
|
||
# --- Set up sample/demo wallets and configuration | ||
RUN mkdir -p --mode=700 ${MINA_CONFIG_DIR}/wallets/store/ \ | ||
|
@@ -99,15 +107,14 @@ RUN mkdir -p --mode=700 ${MINA_CONFIG_DIR}/wallets/store/ \ | |
> ${MINA_CONFIG_DIR}/wallets/store/${PK} \ | ||
&& chmod go-rwx ${MINA_CONFIG_DIR}/wallets/store/${PK} | ||
|
||
# --- Copy artifacts from the 3-builder stage | ||
COPY --from=builder /home/opam/app/* /usr/local/bin/ | ||
COPY --from=builder /home/opam/mina/src/app/rosetta/*.sh /rosetta/ | ||
COPY --from=builder /home/opam/mina/src/app/rosetta/*.conf /rosetta/ | ||
COPY --from=builder /home/opam/mina/src/app/rosetta/README.md /rosetta/ | ||
COPY --from=builder /home/opam/mina/src/app/rosetta/rosetta-cli-config/*.json /rosetta/rosetta-cli-config/ | ||
COPY --from=builder /home/opam/mina/src/app/rosetta/rosetta-cli-config/*.ros /rosetta/rosetta-cli-config/ | ||
COPY --from=builder /home/opam/mina/src/app/archive/*.sql /archive/ | ||
COPY --from=builder /home/opam/mina/genesis_ledgers /genesis_ledgers/ | ||
|
||
# Mina daemon package | ||
# jemalloc is also installed automatically here to match the package dependencies for this $deb_codename | ||
RUN echo "Building image with version $deb_version from repo $deb_release $deb_codename for network $network" \ | ||
&& echo "deb [trusted=yes] http://packages.o1test.net $deb_codename $deb_release" > /etc/apt/sources.list.d/o1.list \ | ||
&& apt-get update --quiet --yes \ | ||
&& apt-get install --quiet --yes --allow-downgrades "mina-$network=$deb_version" "mina-archive=$deb_version" \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
# --- Set up postgres | ||
USER postgres | ||
|
@@ -118,10 +125,10 @@ RUN POSTGRES_VERSION=$(psql -V | cut -d " " -f 3 | sed 's/.[[:digit:]]*$//g') \ | |
# Run as root so it can create /data/postgresql | ||
USER root | ||
RUN POSTGRES_VERSION=$(psql -V | cut -d " " -f 3 | sed 's/.[[:digit:]]*$//g') \ | ||
&& pg_createcluster --start -d /data/postgresql --createclusterconf /rosetta/postgresql.conf ${POSTGRES_VERSION} main | ||
&& pg_createcluster --start -d /data/postgresql --createclusterconf /etc/mina/rosetta/postgresql.conf ${POSTGRES_VERSION} main | ||
|
||
# --- Container workdir, ports, entrypoint, etc. | ||
WORKDIR /rosetta | ||
WORKDIR /etc/mina/rosetta | ||
|
||
EXPOSE 3087 | ||
EXPOSE $MINA_DAEMON_PORT | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this line commented out? If it's unnecessary, let's delete it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is an explanation above to uncomment this line once coinbase fix delegation transaction support