docker compose rework #256

jimmcgaw · 2023-02-12T02:05:44Z

What's in this PR:

A Dockerfile was created for building the leapchat server as a go binary docker image.
A Dockerfile was created for building a frontend image.
Updated the docker compose file to build the frontend image, go server image, as well as the two database images.
Updated the docker-compose file database image steps to include passing the PostgREST config and running the init script in the postgres container.

WIP: have the sql init shell script run in the postgres container when it boots up.

.gitignore

Makefile

elimisteve · 2023-02-12T03:01:30Z

Nice PR! Thank you.

I think I have a permanent solution to both the miniware/ issue and a temporary fix for the React dependency hell issue that's good enough for the foreseeable future.

If I solve the former, shall I just push it to this branch? Or shall I branch off of it so you can take a look?

jimmcgaw · 2023-02-12T03:03:42Z

Nice PR! Thank you.

I think I have a permanent solution to both the miniware/ issue and a temporary fix for the React dependency hell issue that's good enough for the foreseeable future.

If I solve the former, shall I just push it to this branch? Or shall I branch off of it so you can take a look?

Feel free to add whatever to this branch. ⚡

Dockerfile.dev

elimisteve · 2023-02-12T03:21:24Z

@jimmcgaw Just pushed! Got the Go build working. docker compose up now almost works 😄

docker-compose.yml

elimisteve · 2023-02-12T03:27:49Z

@jimmcgaw Just pushed! Got the Go build working. docker compose up now almost works smile

@jimmcgaw Want to take it from here with the dockerization? While I work on dependency hell sheeit.

frontend/package.json

elimisteve · 2023-02-12T07:00:59Z

What do you think is the best way to make it so we can modify LeapChat source files on our host but have the dev server that's running in Docker update when something changes? Shall we attach a volume (that is the folder containing the source code on the host) to the frontend container from the outside?

elimisteve · 2023-02-12T07:08:29Z

The database steps might need tweaking. This is just a starting point for doing local development with docker.

Excellent.

I believe the main reason why the Docker config didn't work in the past is because db/init_sql.sh is never executed inside the postgres container.

Just created an issue for this: #257 ...and added it to this new milestone: https://github.com/cryptag/leapchat/milestone/7

Dockerfile.dev

elimisteve · 2023-02-12T09:03:11Z

I think we need to create Dockerfile.postgrest.dev and have it copy db/postgrest.conf into the postgrest container when building it.

elimisteve · 2023-02-12T09:16:08Z

Created an issue for that: #259

elimisteve · 2023-02-12T11:15:57Z

Alright, this PR and branch are all yours 😅 .

In my mind the last remaining things needed to do here in the milestone I made for dockerization: https://github.com/cryptag/leapchat/milestone/7

jimmcgaw · 2023-02-12T16:33:47Z

I think we need to create Dockerfile.postgrest.dev and have it copy db/postgrest.conf into the postgrest container when building it.

I don't think we need to do that. I create a volume that mounts the /db directory into the container, and a command that runs the postgrest command with the conf file, where it's mounted in the container, when it starts up. If that doesn't work I'll tweak it.

jimmcgaw · 2023-02-12T16:37:57Z

The database steps might need tweaking. This is just a starting point for doing local development with docker.

Excellent.

I believe the main reason why the Docker config didn't work in the past is because db/init_sql.sh is never executed inside the postgres container.

Just created an issue for this: #257 ...and added it to this new milestone: https://github.com/cryptag/leapchat/milestone/7

Yep. I started a volume mount in postgres container that mounts to /docker-entrypoint-initdb.d/, which is the directory that a postgres container will run scripts out of, if they are present, when initialized. I haven't confirmed that this works yet.

jimmcgaw · 2023-02-12T16:40:36Z

What do you think is the best way to make it so we can modify LeapChat source files on our host but have the dev server that's running in Docker update when something changes? Shall we attach a volume (that is the folder containing the source code on the host) to the frontend container from the outside?

For the frontend stuff, yes.

For the go, the ideal workflow isn't clear to me yet. I don't think I've ever worked with Docker with a compiled (ie not interpreted or JIT compiled) language. Any ideas there?

docker-compose.yml

package.json

elimisteve · 2023-02-13T18:34:49Z

docker-compose.yml

    postgres:
        image: postgres:latest
        ports:
-            - 127.0.0.1:5432:5432
+            - 5432:5433


@jimmcgaw I think these should be reversed, yes? To avoid port collisions on the server.

Yep, got these backwards. 🙅

jimmcgaw · 2023-02-13T18:59:52Z

Dockerfile.dev

+
+EXPOSE 8080
+
+CMD ["/leapchat"]


@elimisteve So you're right, npm run dev just does a build to the /build directory. The leapchat binary just serves content directly from there.

I removed the Docker frontend container / file. I don't think it makes sense right here, given the current workflow.

Instead, in the immediate-term, I think we can create a volume in this container that binds build to a directory next to the binary in the container? npm start would then run the watch-build loop and update the build dir in the go container.

@jimmcgaw Removing that other container and mounting build into the Go container sounds perfect.

The only detail I'd add is: instead of putting the leapchat Go binary at /leapchat, let's put it next to the other *.go files just so that leapchat is running in the same environment everywhere: in Docker, on my laptop, and on the production server. I mention that here because this implies putting the build directory right next to leapchat and thus inside the /app/github.com/cryptag/leapchat directory.

Sound good?

EDIT: I expanded on and improved on this idea below.

In that case I think we just need this:

FROM golang:1.19-alpine RUN mkdir -p /app/github.com/cryptag/leapchat/miniware WORKDIR /app/github.com/cryptag/leapchat/ COPY miniware/* miniware/ COPY *.go . COPY go.* . COPY LICENSE.md . RUN go build EXPOSE 8080 CMD ["./leapchat"]

And then to mount ./build as /app/github.com/cryptag/leapchat/build inside the container. (I believe /app/github.com/cryptag/leapchat/build doesn't need to and indeed probably shouldn't exist inside the container, since mounting a volume to that directory would overwrite it or maybe not work?)

Actually, if this works we can make it more like you suggested above, with a simpler directory structure in the container:

FROM golang:1.19-alpine RUN mkdir -p /leapchat/miniware WORKDIR /leapchat COPY miniware/* miniware/ COPY *.go . COPY go.* . COPY LICENSE.md . RUN go build EXPOSE 8080 CMD ["./leapchat"]

…ooting with dbs

elimisteve

Looking pretty great! See comments.

elimisteve · 2023-02-13T18:59:42Z

docker-compose.yml

    postgres:
        image: postgres:latest
        ports:
-            - 127.0.0.1:5432:5432
+            - 5432:5433


elimisteve · 2023-02-13T19:44:56Z

Dockerfile.dev

+
+EXPOSE 8080
+
+CMD ["/leapchat"]


@jimmcgaw Removing that other container and mounting build into the Go container sounds perfect.

The only detail I'd add is: instead of putting the leapchat Go binary at /leapchat, let's put it next to the other *.go files just so that leapchat is running in the same environment everywhere: in Docker, on my laptop, and on the production server. I mention that here because this implies putting the build directory right next to leapchat and thus inside the /app/github.com/cryptag/leapchat directory.

Sound good?

EDIT: I expanded on and improved on this idea below.

elimisteve · 2023-02-13T19:49:51Z

Dockerfile.dev

+
+EXPOSE 8080
+
+CMD ["/leapchat"]


In that case I think we just need this:

FROM golang:1.19-alpine RUN mkdir -p /app/github.com/cryptag/leapchat/miniware WORKDIR /app/github.com/cryptag/leapchat/ COPY miniware/* miniware/ COPY *.go . COPY go.* . COPY LICENSE.md . RUN go build EXPOSE 8080 CMD ["./leapchat"]

And then to mount ./build as /app/github.com/cryptag/leapchat/build inside the container. (I believe /app/github.com/cryptag/leapchat/build doesn't need to and indeed probably shouldn't exist inside the container, since mounting a volume to that directory would overwrite it or maybe not work?)

elimisteve · 2023-02-13T19:52:46Z

Makefile

+	docker compose up -d --build
+
+cleanv:
+	docker compose down -v


(Copied from the email I just sent since this is better context for it --)

How about if I [some time soon, but not in this branch right now] add make release to cut a new (tagged, versioned) release, and make deploy to create a production tarball then scp it to the server?

elimisteve

Oops, I meant for these to be one-off comments, not part of a new review

docker-compose.yml

elimisteve · 2023-02-13T20:05:58Z

Dockerfile.dev

+
+EXPOSE 8080
+
+CMD ["/leapchat"]


Actually, if this works we can make it more like you suggested above, with a simpler directory structure in the container:

FROM golang:1.19-alpine RUN mkdir -p /leapchat/miniware WORKDIR /leapchat COPY miniware/* miniware/ COPY *.go . COPY go.* . COPY LICENSE.md . RUN go build EXPOSE 8080 CMD ["./leapchat"]

leapchat.go

Makefile

elimisteve · 2023-02-13T20:48:35Z

Dockerfile.dev

+
+WORKDIR /app/github.com/cryptag/leapchat/miniware
+
+WORKDIR /app/github.com/cryptag/leapchat


@jimmcgaw I believe we want the 3 lines above into these 2, so that our working directory exists and so we don't have the additional WORKDIR:

RUN mkdir -p /app/github.com/cryptag/leapchat/miniware WORKDIR /app/github.com/cryptag/leapchat

Then I think this PR is ready to merge! Is it working well for you locally?

Oh yeah there needs to be a bit more done: https://github.com/cryptag/leapchat/milestone/7

elimisteve · 2023-02-13T20:55:15Z

For the go, the ideal workflow isn't clear to me yet. I don't think I've ever worked with Docker with a compiled (ie not interpreted or JIT compiled) language. Any ideas there?

The Go binary will need to change so infrequently, and it compiles so quickly, that I say we don't worry about having anything watch the file system for changes to .go files; let's just manually rebuild the Go container when we need to.

Sound good?

bhagwatis · 2023-03-09T14:05:49Z

New to docker and postgrest. I have installed new version of docker. Containers installed posgresql, pgadmin4 and postgrest. Postgrest does not start in the container nor am i able to ping that specific port. Can someone please help me here?

jimmcgaw requested a review from elimisteve February 12, 2023 02:05

elimisteve reviewed Feb 12, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

elimisteve reviewed Feb 12, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved