Merge pull request #35 from thin-edge/feat-support-zips

feat(container-group): Support zip and gzip formats

README.md

@@ -14,7 +14,7 @@ The instructions assume that you are using thin-edge.io >= 1.0.0
 * A service called `tedge-container-monitor`. This provides the monitoring of the containers
 * The following software management plugins which is called when installing and removing containers/container groups via Cumulocity IoT
     * `container` - Deploy a single container (`docker run xxx` equivalent)
-    * `container-group` - Deploy one or more container as defined by a `docker-compose.yaml` file (`docker compose up` equivalent)
+    * `container-group` - Deploy one or more container as defined by a `docker-compose.yaml` file (`docker compose up` equivalent), or an archive (gzip or zip)
 
 **Note**
 
@@ -46,7 +46,7 @@ The following linux package formats are provided on the releases page and also i
 The following features are supported by the plugin:
 
 * Install/remove containers via the Cumulocity IoT software interface
-* Install multiple containers as one group using a `docker-compose.yaml` file
+* Install multiple containers as one group using a `docker-compose.yaml` file or an archive container a `docker-compose.yaml` file
 * Monitor container states (e.g. up/down) via Cumulocity IoT Services (only supported from tedge >= 0.10.0)
 * Download container images via Cumulocity IoT binaries if a URL is provided
 * Support for multiple container engines (docker, podman, nerdctl)
@@ -71,7 +71,7 @@ The software package properties are also describe below:
 
 ### Install/remove a `container-group`
 
-A `container-group` is the name given to deploy a `docker-compose.yaml` file. A docker compose file allows use to deploy multiple containers/networks/volumes and allows you maximum control over how the container is started. This means you can create a complex setup of persisted volumes, isolated networks, and also facilitate communication between containers. Check out the [docker compose documentation](https://docs.docker.com/compose/compose-file/) for more details on how to write your own service definition.
+A `container-group` is the name given to deploy a `docker-compose.yaml` file or an archive (zip or gzip file) with the `docker-compose.yaml` file at the root level of the archive. A docker compose file allows use to deploy multiple containers/networks/volumes and allows you maximum control over how the container is started. This means you can create a complex setup of persisted volumes, isolated networks, and also facilitate communication between containers. Check out the [docker compose documentation](https://docs.docker.com/compose/compose-file/) for more details on how to write your own service definition.
 
 The software package properties are also describe below:
 
@@ -302,21 +302,27 @@ To run the tests you will need to have python3 >> 3.9 installed on your syste
    C8Y_PASSWORD="mypassword"
    ```
 
-2. Startup the test setup, and bootstrap it
+2. Build the software management plugin
+
+   ```
+   just build
+   ```
+
+3. Startup the test setup, and bootstrap it
 
    ```
    just up
 
    just bootstrap
    ```
 
-3. Setup the python3 virtual environment and install the test dependencies
+4. Setup the python3 virtual environment and install the test dependencies
 
    ```
    just venv
    ```
 
-4. Run the RobotFramework tests
+5. Run the RobotFramework tests
 
    ```
    just test

images/Dockerfile

@@ -11,9 +11,8 @@ RUN install -m 0755 -d /etc/apt/keyrings \
 RUN apt-get update \
     && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
         docker-ce-cli \
-        docker-compose-plugin
-    # && groupadd docker \
-    # && usermod -aG docker
+        docker-compose-plugin \
+        unzip
 
 COPY ./images/container-env /etc/tedge-container-plugin/env
 

src/container-group

@@ -171,13 +171,77 @@ case "$COMMAND" in
             $COMPOSE_CLI down ||:
         fi
 
+        if [ -n "$LOCAL_MODULE_PATH" ]; then
+            log "Removing files in $LOCAL_MODULE_PATH"
+            rm -Rf "${LOCAL_MODULE_PATH:?}/"*
+        fi
+
+        # Try getting the mime-type from the file
+        # Use file if available, otherwise try to validate
+        # the file using various tools to guess the correct type.
+        # Note: alpine does not have "file" installed out of the box
+        FILE_TYPE="unknown"
+        if command -v file >/dev/null 2>&1; then
+            FILE_TYPE=$(file -db --mime-type "$FILE")
+        elif unzip -t "$FILE" >/dev/null 2>&1; then
+            FILE_TYPE="application/zip"
+        elif tar tf "$FILE" >/dev/null 2>&1; then
+            FILE_TYPE="application/gzip"
+        fi
+
+        EXTRA_OPIONS=""
+
+        log "Detected file type: $FILE_TYPE ($FILE)"
+        case "$FILE_TYPE" in
+            application/zip)
+                if ! command -v unzip >/dev/null 2>&1; then
+                    log "unzip is not installed on your system. It is required to handle zip files"
+                    exit 1
+                fi
+
+                if ! unzip -l "$FILE" "docker-compose.yaml" >/dev/null 2>&1 && ! unzip -l "$FILE" "docker-compose.yml" >/dev/null 2>&1; then
+                    log "No docker-compose file found in the zip archive."
+                    exit "$EXIT_FAILURE"
+                fi
+
+                log "Extracting zip file to $LOCAL_MODULE_PATH"
+                unzip -o "$FILE" -d "$LOCAL_MODULE_PATH"
+                EXTRA_OPIONS="$EXTRA_OPIONS --build"
+                ;;
+
+            application/gzip)
+                if ! command -v tar >/dev/null 2>&1; then
+                    log "tar is not installed on your system. It is required to handle gzip files"
+                    exit 1
+                fi
+                if ! tar tf "$FILE" "docker-compose.yaml" >/dev/null 2>&1 && ! tar tf "$FILE" "docker-compose.yml" >/dev/null 2>&1; then
+                    log "No docker-compose file found in the gzip archive."
+                    exit "$EXIT_FAILURE"
+                fi
+
+                log "Extracting gzip file to $LOCAL_MODULE_PATH"
+                tar xzvf "$FILE" -C "$LOCAL_MODULE_PATH" >&2
+                EXTRA_OPIONS="$EXTRA_OPIONS --build"
+                ;;
+            text/plain)
+                log "Copying file to $LOCAL_MODULE_PATH"
+                cp "$FILE" "$LOCAL_MODULE_PATH/docker-compose.yaml"
+                ;;
+            *)
+                # let's assume it is a text file and try anyway, docker compose
+                # will fail if it is not in the correct format
+                log "Copying file (of unrecognized file type $FILE_TYPE) to $LOCAL_MODULE_PATH"
+                cp "$FILE" "$LOCAL_MODULE_PATH/docker-compose.yaml"
+                ;;
+        esac
+
         log "Storing module version: $VERSION_FILE"
-        cp "$FILE" "$LOCAL_MODULE_PATH/docker-compose.yaml"
         echo "${MODULE_VERSION}" > "$VERSION_FILE"
 
         EXIT_CODE="$EXIT_OK"
         log "Deploying project from path $LOCAL_MODULE_PATH"
-        if ! $COMPOSE_CLI up --detach --remove-orphans; then
+        # shellcheck disable=SC2086
+        if ! $COMPOSE_CLI up --detach --remove-orphans $EXTRA_OPIONS; then
             log "Failed to start project"
 
             EXIT_CODE="$EXIT_FAILURE"

tests/data/apps/app1/Dockerfile

@@ -0,0 +1,2 @@
+FROM nginx
+COPY static /usr/share/nginx/html

tests/data/apps/app1/docker-compose.yaml

@@ -0,0 +1,13 @@
+version: "3"
+services:
+  app1:
+    build:
+      dockerfile: Dockerfile
+      context: .
+    networks:
+      - tedge
+
+networks:
+  tedge:
+    name: tedge
+    external: true

tests/data/apps/app1/static/index.html

@@ -0,0 +1,25 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+    <title>My Custom Web Application</title>
+    <style>
+        html {
+            color-scheme: light dark;
+        }
+
+        body {
+            width: 35em;
+            margin: 0 auto;
+            font-family: Tahoma, Verdana, Arial, sans-serif;
+        }
+    </style>
+</head>
+
+<body>
+    <h1>My Custom Web Application</h1>
+    <p>If you see this page, the nginx web server is successfully installed and
+        working. Further configuration is required.</p>
+</body>
+
+</html>

tests/data/apps/app2/Dockerfile

@@ -0,0 +1,2 @@
+FROM nginx
+COPY static /usr/share/nginx/html

tests/data/apps/app2/docker-compose.yaml

@@ -0,0 +1,13 @@
+version: "3"
+services:
+  app2:
+    build:
+      dockerfile: Dockerfile
+      context: .
+    networks:
+      - tedge
+
+networks:
+  tedge:
+    name: tedge
+    external: true

tests/data/apps/app2/static/index.html

@@ -0,0 +1,25 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+    <title>My Custom Web Application</title>
+    <style>
+        html {
+            color-scheme: light dark;
+        }
+
+        body {
+            width: 35em;
+            margin: 0 auto;
+            font-family: Tahoma, Verdana, Arial, sans-serif;
+        }
+    </style>
+</head>
+
+<body>
+    <h1>My Custom Web Application</h1>
+    <p>If you see this page, the nginx web server is successfully installed and
+        working. Further configuration is required.</p>
+</body>
+
+</html>

tests/data/apps/build.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+set -e
+#
+# Package some example docker compose as gzip and zip packages
+#
+
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+pushd "$SCRIPT_DIR" >/dev/null ||:
+
+(cd app1 && tar czvf ../app1.tar.gz docker-compose.yaml Dockerfile static/*)
+(cd app2 && zip ../app2.zip docker-compose.yaml Dockerfile static/*)
+
+popd ||:

tests/operations.robot

@@ -25,6 +25,11 @@ Install container-group package
     Should Contain    ${operation.to_json()["c8y_Command"]["result"]}    Welcome to nginx
     Cumulocity.Should Have Services    name=nginx@nginx    service_type=container-group    status=up
 
+Install container-group with multiple files
+    [Template]    Install container-group file
+    app1    1.0.1    app1    ${CURDIR}/data/apps/app1.tar.gz
+    app2    1.2.3    app2    ${CURDIR}/data/apps/app2.zip
+
 Uninstall container-group
     ${operation}=     Cumulocity.Uninstall Software    {"name": "nginx", "version": "1.0.0", "softwareType": "container-group"}
     Operation Should Be SUCCESSFUL    ${operation}
@@ -49,3 +54,14 @@ Uninstall container package
 
 Suite Setup
     Set Main Device
+
+Install container-group file
+    [Arguments]    ${package_name}    ${package_version}    ${service_name}    ${file}
+    ${binary_url}=    Cumulocity.Create Inventory Binary    ${package_name}    container-group    file=${file}
+    ${operation}=    Cumulocity.Install Software    {"name": "${package_name}", "version": "${package_version}", "softwareType": "container-group", "url": "${binary_url}"}
+    Operation Should Be SUCCESSFUL    ${operation}
+    Device Should Have Installed Software    {"name": "${package_name}", "version": "${package_version}", "softwareType": "container-group"}
+    ${operation}=    Cumulocity.Execute Shell Command    wget -O- ${service_name}:80
+    Operation Should Be SUCCESSFUL    ${operation}
+    Should Contain    ${operation.to_json()["c8y_Command"]["result"]}    My Custom Web Application
+    Cumulocity.Should Have Services    name=${package_name}@${service_name}    service_type=container-group    status=up