rootless docker + podman

docs/getting-started.md

@@ -97,10 +97,30 @@ Confirm that Docker installed correctly by opening a terminal and running `docke
 
 To avoid writing sudos you may consider [adding yourself to docker group](https://docs.docker.com/install/linux/linux-postinstall/)
 
-Keep in mind that if you do so, you can now run containers without sudo and containers give you super user access to the computer.
+Keep in mind that if you do so, you can now run containers without sudo and containers give you or anyone who gains access to your account super user access to the computer.
 
 :::
 
+### Rootless Docker
+
+Instead of above installation, on Linux you can run Docker as a non-root user. This requires that your system has [certain programs and configurations set up in advance](https://docs.docker.com/engine/security/rootless/#prerequisites) by the system administrator. Since running Docker in rootless mode limits risks for system security, convincing your IT admin to cooperate to the extent required for rootless Docker may be viable alternative if you do not have administrative privileges yourself.
+
+If your system is set up correctly, you can possibly run locally available installation script dockerd-rootless-setuptool.sh or download it from [https://get.docker.com/rootless](https://get.docker.com/rootless). The script will inform you of any missing requirements, if there are any. The script will also tell you how to control docker service, and what settings to set or change in your environment (PATH, DOCKER_HOST).
+
+See also [known limitations](https://docs.docker.com/engine/security/rootless/#known-limitations).
+
+Do note that while running Docker rootless does limit some security risks to your system, it just adds one hurdle for potential malicious attacker (and in any case, system staying secure might not sound so great if you end up "only" losing your user data).
+
+### Podman
+
+Alternatively, your system may already have [Podman](https://podman.io/) installed (or you can [install it yourself](https://podman.io/docs/installation)). Podman is another containerization framework, functioning as a drop-in replacement for Docker to a high degree - often you can just replace command docker with command podman. If you intend to work later with Kubernetes, Podman also offers some extra conveniences, but on the other hand Podman does not support Docker Swarm.
+
+This course should be doable with `alias docker=podman`. Easiest way to make this permanent is `echo "alias docker=podman" >> .bashrc`. Podman does not use default registry, while Docker uses docker.io by default. You will need to remember define registry with your commands (eg, `docker run docker.io/nginx`) or you need to configure your default registry (`mkdir $HOME/.config/containers && echo "unqualified-search-registries = ['docker.io']" >> $HOME/.config/containers/registries.conf`)
+
+If `podman info | grep graphDriverName` tells you are using vfs as your storage driver, prepare for _very_ slow and large builds. Requesting (or installing yourself) and then using fuse-overlayfs or native overlay (in recent Ubuntu and Debian package containers-storage) would be a good idea. (Or possibly changing default configuration with $HOME/.config/containers/storage.conf might be enough. [source](https://blog.abysm.org/2023/06/switching-system-wide-default-storage-driver-from-vfs-to-overlayfs-for-podman-on-debian-bookworm/))
+
+Some of the Podman output may slightly differ from example Docker output in the material.
+
 ## Deadline
 
 The sign up for ECTS credits and the course ends 16.6.2024! After that course is locked and submissions can no longer be made or credits earned. As the certificate is received through submissions, you have to submit everything before the course ends. More details under completion and after each part.

docs/part-1/section-2.md

@@ -77,6 +77,12 @@ $ docker attach looper
 
 Now you have process logs (STDOUT) running in two terminals. Now press control+c in the attached window. The container is stopped because the process is no longer running.
 
+:::tip Podman
+
+If you are using Podman, depending on your version, control+c with --no-stdin mentioned below might not work.
+
+:::
+
 If we want to attach to a container while making sure we don't close it from the other terminal we can specify to not attach STDIN with `--no-stdin` option. Let's start the stopped container with `docker start looper` and attach to it with `--no-stdin`.
 
 Then try control+c.
@@ -140,7 +146,7 @@ $ docker exec -it looper bash
 
 From the `ps aux` listing we can see that our `bash` process got PID (process ID) of 64.
 
-Now that we're inside the container it behaves as you'd expect from Ubuntu, and we can exit the container with `exit` and then either kill or stop the container.
+Now that we're inside the container it behaves as you'd expect from Ubuntu, and we can exit the container with `exit` or control+d and then either kill or stop the container.
 
 Our looper won't stop for a SIGTERM signal sent by a stop command. To terminate the process, stop follows the SIGTERM with a SIGKILL after a grace period. In this case, it's simply faster to use kill.
 

docs/part-1/section-3.md

@@ -26,6 +26,12 @@ $ docker search hello-world
   ...
 ```
 
+:::tip Podman
+
+If you are using Podman, use --compatible to see stars, official and automated columns. For explanation, see `man podman-search`.
+
+:::
+
 Let's examine the list.
 
 The first result, `hello-world`, is an official image. [Official images](https://docs.docker.com/docker-hub/official_images/) are curated and reviewed by Docker, Inc. and are usually actively maintained by the authors. They are built from repositories in the [docker-library](https://github.com/docker-library).
@@ -40,7 +46,7 @@ There are also other Docker registries competing with Docker Hub, such as [Quay]
 
 `docker pull quay.io/nordstrom/hello-world`
 
-So, if the host's name (here: `quay.io`) is omitted, it will pull from Docker Hub by default.
+So, if the alternative registry's name (here: `quay.io`) is omitted, it will pull from Docker Hub by default.
 
 NOTE: Trying the above command may fail giving manifest errors as the default tag latest is not present in quay.io/nordstrom/hello-world image. Specifying a correct tag for a image will pull the image without any errors, for ex.
 `docker pull quay.io/nordstrom/hello-world:2.0`
@@ -206,6 +212,14 @@ If you're now getting "/bin/sh: ./hello.sh: not found" and you're using Windows
 
 :::
 
+:::tip can't stat
+
+If you are running rootless docker and build process gives error saying "can't stat...", you may try removing old images.
+
+If you keep getting this error with future exercises, this may be a significant problem with some later exercises. It may be worthwhile to check if you could do the course with standard Docker or with Podman instead of rootless Docker.
+
+:::
+
 Now executing the application is as simple as running `docker run hello-docker`. Try it!
 
 During the build we see from the output that there are three steps: [1/3], [2/3] and [3/3]. The steps here represent [layers](https://docs.docker.com/build/guide/layers/) of the image so that each step is a new layer on top of the base image (alpine:3.19 in our case).
@@ -378,7 +392,7 @@ Try `docker run devopsdockeruh/simple-web-service:alpine hello`. The application
 
 In this exercise create a Dockerfile and use FROM and CMD to create a brand new image that automatically runs `server`.
 
-The Docker documentation [CMD](https://docs.docker.com/engine/reference/builder/#cmd) says a bit indirectly that if a image has ENTRYPOINT defined, CMD is used to define it the default arguments.
+The Docker documentation [CMD](https://docs.docker.com/engine/reference/builder/#cmd) says a bit indirectly that if an image has ENTRYPOINT defined, CMD is used to define the default arguments for it.
 
 Tag the new image as "web-server"
 

docs/part-1/section-4.md

@@ -16,30 +16,30 @@ $ docker run -it ubuntu:22.04
 ..and, as we already know, curl is not installed - let's add `curl` with `apt-get` again.
 
 ```console
-$ apt-get update && apt-get install -y curl
-$ curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp
+# apt-get update && apt-get install -y curl
+# curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp
 ```
 
 At some point, you may have noticed that _sudo_ is not installed either, but since we are _root_ we don't need it.
 
 Next, we will add permissions and run the downloaded binary:
 
 ```console
-$ chmod a+rx /usr/local/bin/yt-dlp
-$ yt-dlp
+# chmod a+rx /usr/local/bin/yt-dlp
+# yt-dlp
 /usr/bin/env: 'python3': No such file or directory
 ```
 
 Okay, [documentation](https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#dependencies) mentions that Python 3.8 or later is needed to run yt-dlp. So let us install that:
 
 ```console
-$ apt-get install -y python3
+# apt-get install -y python3
 ```
 
 We can now try to run the app again:
 
 ```console
-$ yt-dlp
+# yt-dlp
 
   Usage: yt-dlp [OPTIONS] URL [URL...]
 
@@ -259,11 +259,17 @@ $ docker diff determined_elion
 Let's try `docker cp` command to copy the file from the container to the host machine. We should use quotes now since the filename has spaces.
 
 ```console
-$ docker cp "determined_elion://mydir/Welcome to Kumpula campus! ｜ University of Helsinki [DptFY_MszQs].mp4" .
+$ docker cp "determined_elion:/mydir/Welcome to Kumpula campus! ｜ University of Helsinki [DptFY_MszQs].mp4" .
 ```
 
 And now we have our file locally and we can watch it if the machine has a suitable player installed. Sadly, the use of `docker cp` is not proper to fix our issue. In the next section, we will improve this.
 
+:::tip Podman
+
+If you are using Podman, you may also need to escape `[` in file name. (ie `podman cp "determined_elion:/mydir/Welcome to Kumpula campus! ｜ University of Helsinki \[DptFY_MszQs].mp4" .`)
+
+:::
+
 ## Improved curler
 
 With `ENTRYPOINT` we can make the curler of the [Exercise 1.7.](/part-1/section-3#exercises-17---18) more flexible.

docs/part-1/section-5.md

@@ -37,7 +37,7 @@ This course does not provide an in-depth exploration of inter-program communicat
 
 - Sending messages: Programs can send messages to [URL](https://en.wikipedia.org/wiki/URL) addresses such as this: http://127.0.0.1:3000 where HTTP is the [_protocol_](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol), 127.0.0.1 is an IP address, and 3000 is a [_port_](<https://en.wikipedia.org/wiki/Port_(computer_networking)>). Note the IP part could also be a [_hostname_](https://en.wikipedia.org/wiki/Hostname): 127.0.0.1 is also called [_localhost_](https://en.wikipedia.org/wiki/Localhost) so instead you could use http://localhost:3000.
 
-- Receiving messages: Programs can be assigned to listen to any available port. If a program is listening for traffic on port 3000, and a message is sent to that port, the program will receive and possibly process it.
+- Receiving messages: Programs can be assigned to listen to any available port. If a program is listening for traffic on port 3000, and a message is sent to that port, the program will receive and possibly process it. Port numbers below 1024 may be unavailable for users with normal privileges.
 
 The address _127.0.0.1_ and hostname _localhost_ are special ones, they refer to the machine or container itself, so if you are on a container and send a message to _localhost_, the target is the same container. Similarly, if you are sending the request from outside of a container to _localhost_, the target is your machine.
 

docs/part-1/section-6.md

@@ -156,6 +156,16 @@ TIPS:
 
 :::
 
+:::tip can't stat
+
+If you are running rootless docker and have been struggling this far with build process sometimes giving error "can't stat...", following exercise 1.14 will present a major challenge.
+
+If this error is still a problem for you, possible workarounds:
+ - Jump to next section ("Publishing projects"), and upload your images to registry. Pulling one image and building another one may work.
+ - Part 2 of this course, using docker-compose, may allow building several images at the same time, see exercise 2.3.
+
+:::
+
 :::caution Mandatory Exercise 1.14: Environment
 
 Start both the frontend and the backend with the correct ports exposed and add [ENV](https://docs.docker.com/reference/dockerfile/#env) to Dockerfile with the necessary information from both READMEs

docs/part-2/section-1.md

@@ -6,6 +6,26 @@ Even with a simple image, we've already been dealing with plenty of command line
 
 Next we will switch to a tool called [Docker Compose](https://docs.docker.com/compose/) to manage these. Docker Compose used to be a separate tool but now it is integrated into Docker and can be used like the rest of the Docker commands.
 
+:::tip Rootless Docker
+
+If you are using Docker without root privileges, your system might not have Docker Compose installed. If this is the case, you can [install it manually yourself](https://docs.docker.com/compose/install/linux/#install-the-plugin-manually).
+
+:::
+
+:::tip Podman
+
+For small scale orchestration Podman supports Podman Compose and Docker Compose, which you may or may not already have installed. Since Docker Compose is reference for everyone else, latest versions of Podman should use it by default if both Podman Compose and Docker Compose are available. You can read more about relationships between these projects [here](https://www.redhat.com/en/blog/podman-compose-docker-compose).
+
+For purposes of this course, it is better to use Docker Compose. If you do not already have it installed, you can install it manually yourself as a [standalone version](https://docs.docker.com/compose/install/standalone/) (or possibly from your operating system packages, for example `apt install --no-install-recommends docker-compose podman-docker`). Depending on your software versions and how you installed it, you may have to call `docker-compose` instead of `docker compose` (or `podman compose`).
+
+For docker-compose to work with Podman, you need a socket: `systemctl --user enable --now podman.socket`, check path with `podman info | grep -i remotesocket -A2` and then export required environment variable: for example `export DOCKER_HOST=unix:///run/user/$UID/podman/podman.sock`. If you want to make Compose always available, add these to your .bashrc. 
+
+If necessary, Podman Compose can be found [here](https://github.com/containers/podman-compose). Read README.md for installation instructions. It is more lightweight and does not require socket, but has not been tested with this course. Do note that even if you install both, you should only use one of them, they are not interchangeable.
+
+If you are not forced to use Compose, check also [pods](https://developers.redhat.com/blog/2019/01/15/podman-managing-containers-pods) (see also man podman-pod), [quadlets](https://www.redhat.com/en/blog/quadlet-podman) (Podman 4.4+) or Kubernetes. (This is not relevant for this course.)
+
+:::
+
 Docker Compose is designed to simplify running multi-container applications using a single command.
 
 Assume that we are in the folder where we have our Dockerfile with the following content:

docs/part-2/section-2.md

@@ -144,6 +144,12 @@ $ docker compose port --index 3 whoami 8000
   0.0.0.0:32768
 ```
 
+:::tip Podman
+
+If you are using Podman, `podman ps` will list also ports.
+
+:::
+
 We can now curl from these ports:
 
 ```console
@@ -186,6 +192,14 @@ $ curl localhost:80
   </html>
 ```
 
+:::tip Rootless Docker and Podman
+
+If you are using rootless Docker, or Podman (Podman runs by default also rootless), you may be unable to use port numbers below 1024. If this is the case, you could for example use 8888:80 for port in your docker-compose.yml and then `curl localhost:8888`.
+
+Note also that for both rootless Docker and for Podman, socket path and possibly name is different. You can see your socket information with `echo $DOCKER_HOST`.
+
+:::
+
 It's "working", but the Nginx just doesn't know which service we want. The `nginx-proxy` works with two environment variables: `VIRTUAL_HOST` and `VIRTUAL_PORT`. `VIRTUAL_PORT` is not needed if the service has `EXPOSE` in it's Docker image. We can see that `jwilder/whoami` sets it: <https://github.com/jwilder/whoami/blob/master/Dockerfile#L9>
 
 - Note: Mac users with the M1 processor you may see the following error message: `runtime: failed to create new OS thread`. In this case you can use the Docker Image `ninanung/nginx-proxy` instead which offers a temporary fix until `jwilder/nginx-proxy` is updated to support M1 Macs.
@@ -276,3 +290,14 @@ This exercise was created with [Sasu Mäkinen](https://github.com/sasumaki)
 Please return the used commands for this exercise.
 
 :::
+
+:::tip Rootless Docker and Podman
+
+If you are using rootless Docker, or Podman (Podman runs by default also rootless), you may be unable to use port numbers below 1024. If this is the case, above exercise may require extra effort.
+
+Like earlier, you could for example use 8888:80 for port in your docker-compose.yml. However, calculator/src/commons.js has hardcoded address (with default port, ie 80) which will be a problem. You will have to edit address to include suitable port, and then build calculator image yourself, instead of using provided one.
+
+Note again that for both rootless Docker and for Podman, socket path and possibly name is different. You can see your socket information with `echo $DOCKER_HOST`.
+
+:::
+

docs/part-2/section-3.md

@@ -162,7 +162,7 @@ services:
     volumes:
       - database:/var/lib/postgresql/data
   redmine:
-    image: redmine:4.1-alpine
+    image: redmine:5.1-alpine
     environment:
       - REDMINE_DB_POSTGRES=db
       - REDMINE_DB_PASSWORD=example
@@ -260,6 +260,14 @@ TIPS:
 
 :::
 
+:::tip Rootless Docker and Podman
+
+In case you do not have root permissions (ie, are using rootless Docker and possibly Podman), following exercise 2.7 is very likely to create a directory which (including contents) you can not remove yourself, due to ownership and permissions. As such, convenient location mentioned in the exercise is *not* `./database` (at least initially), instead you might consider `/tmp/database` (which might get emptied automatically by your operating system, either periodically or during system boot). Note there is no `.` in front of `/tmp/database`.
+
+Read carefully "Arbitrary --user Notes" from Postgres image documentation for hints how to actually solve this problem. This may require building your own image based on Postgres image.
+
+:::
+
 :::info Exercise 2.7
 
 Postgres image uses a volume by default. Define manually a volume for the database in a convenient location such as in `./database` so you should use now a [bind mount](https://docs.docker.com/storage/bind-mounts/). The image [documentation](https://github.com/docker-library/docs/blob/master/postgres/README.md#where-to-store-data) may help you with the task.
@@ -414,3 +422,9 @@ Nmap done: 1 IP address (1 host up) scanned in 1.28 seconds
 ```
 
 :::
+
+:::tip Rootless Docker and Podman
+
+If nmap gives an error about not being permitted to open raw socket, you may want to check in what context [nmap documentation](https://nmap.org/book/man-misc-options.html) mentions raw socket privileges.
+
+:::

docs/part-3/section-2.md

@@ -220,3 +220,9 @@ Submit the Dockerfile and the final version of your script.
   See [Part 1](/part-1/section-4) for more.
 
 :::
+
+:::tip Rootless Docker and Podman
+
+If you are using Podman or rootless Docker, remember your socket path from earlier parts of this material.
+
+:::