Merge pull request #11 from janpreet/feature/keybase

Feature/keybase

CICD_BEST_PRACTICES.md

@@ -0,0 +1,169 @@
+# Kado CI/CD Best Practices
+
+This guide provides best practices for integrating Kado and Keybase into your CI/CD pipeline, especially when running inside containers.
+
+## Table of Contents
+
+1. [Mounting Directories](#mounting-directories)
+2. [Keybase Integration](#keybase-integration)
+3. [Security Considerations](#security-considerations)
+4. [CI/CD Pipeline Configuration](#cicd-pipeline-configuration)
+5. [Best Practices](#best-practices)
+6. [Troubleshooting](#troubleshooting)
+
+## Mounting Directories
+
+To allow Kado to access your configuration files, templates, and other resources, you need to mount the relevant directories from your host system to the container.
+
+### Docker Run Example
+
+```bash
+docker run -v $(pwd):/workspace ghcr.io/janpreet/kado:latest kado [command]
+```
+
+This command mounts the current directory to /kado-workspace in the container.
+
+### Docker Compose Example
+
+```yaml
+yamlCopyversion: '3'
+services:
+  kado:
+    image: ghcr.io/janpreet/kado:latest
+    volumes:
+      - ./:/workspace
+    command: kado [command]
+```
+
+### CI/CD Configuration
+In your CI/CD pipeline, ensure that your job checks out the repository and mounts it to the container:
+
+```yaml
+name: Deploy Infrastructure
+
+on:
+  push:
+    branches: [ main ]
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    container: 
+      image: your-registry/kado:latest
+      volumes:
+        - ${{ github.workspace }}:/kado-workspace
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Keybase
+      run: |
+        echo "${{ secrets.KEYBASE_PAPERKEY }}" | keybase oneshot
+        kado keybase link
+      env:
+        KEYBASE_PAPERKEY: ${{ secrets.KEYBASE_PAPERKEY }}
+    - name: Deploy with Kado
+      run: |
+        cd /kado-workspace
+        kado set cluster.yaml
+```
+
+## Keybase Integration
+
+### Setting Up Keybase in CI/CD
+
+1. Generate a paper key for your Keybase account.
+2. Store the paper key securely in your CI/CD platform's secret management system.
+3. In your CI/CD job, use the paper key to authenticate Keybase:
+
+```yaml
+job_name:
+  image: your-registry/kado:latest
+  script:
+    - echo $KEYBASE_PAPERKEY | keybase oneshot
+    - kado keybase link
+    # Your Kado commands here
+```
+
+### Using Keybase Notes in Templates
+
+Reference Keybase notes in your templates using the `{{keybase:note:note_name}}` syntax:
+
+```hcl
+pm_user = {{keybase:note:proxmox_api_key}}
+pm_password = {{keybase:note:secret_token}}
+```
+
+## Security Considerations
+
+1. **Paper Key Security**: Never expose your Keybase paper key in logs or non-secure storage.
+2. **Ephemeral Sessions**: Use `keybase oneshot` to create temporary Keybase sessions.
+3. **Least Privilege**: Use a Keybase account with minimal necessary permissions for CI/CD.
+4. **Secure Note Storage**: Store sensitive information in Keybase notes, not in your codebase.
+
+## CI/CD Pipeline Configuration
+
+### Example GitHub Actions Workflow
+
+```yaml
+name: Deploy Infrastructure
+
+on:
+  push:
+    branches: [ main ]
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    container: your-registry/kado:latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Keybase
+      run: |
+        echo "${{ secrets.KEYBASE_PAPERKEY }}" | keybase oneshot
+        kado keybase link
+      env:
+        KEYBASE_PAPERKEY: ${{ secrets.KEYBASE_PAPERKEY }}
+    - name: Deploy with Kado
+      run: kado set cluster.yaml
+```
+
+## Best Practices
+
+1. **Configuration Management**:
+   - Use `.kd` files for defining beads.
+   - Keep sensitive data in Keybase notes, referenced in templates.
+
+3. **Testing**:
+   - Implement a test stage in your CI/CD pipeline using `kado -debug`.
+   - Validate templates and configurations before deployment.
+
+4. **Logging and Monitoring**:
+   - Enable debug logging in CI/CD for troubleshooting.
+   - Monitor Keybase activity for any suspicious actions.
+
+5. **Secret Rotation**:
+   - Regularly rotate your Keybase paper key.
+   - Update Keybase notes with new credentials periodically.
+
+6. **Error Handling**:
+   - Implement proper error handling in your CI/CD scripts.
+   - Set up notifications for pipeline failures.
+
+## Troubleshooting
+
+1. **Keybase Authentication Issues**:
+   - Ensure the paper key is correctly stored in CI/CD secrets.
+   - Check Keybase logs for authentication errors.
+
+2. **Template Processing Errors**:
+   - Verify that all referenced Keybase notes exist.
+   - Check for syntax errors in your templates.
+
+3. **Container Issues**:
+   - Ensure all required tools are installed and accessible in the container.
+   - Verify the container has necessary permissions to execute Kado and Keybase.
+
+4. **Pipeline Failures**:
+   - Review CI/CD logs for specific error messages.
+   - Test Kado commands locally to replicate issues.
+
+By following these best practices and guidelines, you can effectively integrate Kado and Keybase into your CI/CD pipeline, ensuring secure and efficient infrastructure management.

Dockerfile

@@ -1,15 +1,28 @@
-FROM alpine:3.18
+FROM debian:bullseye-slim
 
 ARG TERRAFORM_VERSION=1.9.3
-ARG ANSIBLE_VERSION=10.2.0
+ARG ANSIBLE_VERSION=2.6.13
 
-RUN apk add --no-cache \
+RUN apt-get update && apt-get install -y --no-install-recommends \
     bash \
     curl \
-    python3 \
-    py3-pip \
-    aws-cli \
-    && pip3 install --no-cache-dir --upgrade pip
+    gnupg \
+    unzip \
+    python3-pip \
+    awscli \
+    libayatana-appindicator3-1 \
+    fuse \
+    psmisc \
+    lsof \
+    procps \
+    libasound2 \
+    libnss3 \
+    libxss1 \
+    libxtst6 \
+    libgtk-3-0 \
+    && pip3 install --no-cache-dir --upgrade pip \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
 
 RUN curl -LO "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip" \
     && unzip terraform_${TERRAFORM_VERSION}_linux_amd64.zip \
@@ -21,11 +34,17 @@ RUN curl -L -o /usr/local/bin/opa https://openpolicyagent.org/downloads/latest/o
 
 RUN pip3 install --no-cache-dir ansible==${ANSIBLE_VERSION}
 
+RUN curl -s https://keybase.io/docs/server_security/code_signing_key.asc | gpg --import \
+    && curl -O https://prerelease.keybase.io/keybase_amd64.deb \
+    && dpkg -i keybase_amd64.deb \
+    && apt-get install -f \
+    && rm keybase_amd64.deb
+
 COPY kado /usr/local/bin/kado
 
 ENV PATH="/opt/venv/bin:$PATH"
 
 WORKDIR /workspace
 
 #ENTRYPOINT ["tail", "-f", "/dev/null"]
-ENTRYPOINT ["/usr/local/bin/kado"]
+ENTRYPOINT ["/usr/local/bin/kado"]

README.md

@@ -16,6 +16,7 @@ Kado is a modular configuration management tool designed to streamline and autom
   - [Terraform Bead](#terraform-bead)
   - [OPA Bead](#opa-bead)
   - [Terragrunt Bead](#terragrunt-bead)
+- [Keybase Integration](#keybase-integration)
 - [Usage](#usage)
   - [Commands](#commands)
   - [Getting Started](#getting-started)
@@ -90,7 +91,7 @@ Example `vm.tfvars.tmpl`:
 aws_region       = "{{.Get "aws.s3.region"}}"
 pm_api_url       = "{{.Get "proxmox.api_url"}}"
 pm_user          = "{{.Env "PM_USER"}}"
-pm_password      = "{{.Env "PM_PASSWORD"}}"
+pm_password      = "{{ keybase:note:secret_token }}"
 vm_roles = {
   master       = {{.Get "proxmox.vm.roles.master"}}
   worker       = {{.Get "proxmox.vm.roles.worker"}}
@@ -183,16 +184,51 @@ bead "terragrunt" {
 }
 ```
 
+## Keybase Integration
+
+Kado integrates with Keybase to provide secure storage and referencing of sensitive information within your infrastructure configurations.
+
+### Keybase Commands
+
+- `kado keybase link`: Links your Keybase account with Kado.
+- `kado keybase note create <note_name>`: Creates a new encrypted note in Keybase.
+- `kado keybase note list`: Lists all stored notes.
+- `kado keybase note view <note_name>`: Displays the content of a specific note.
+- `kado keybase note share <note_name> <keybase_username>`: Shares a note with another Keybase user.
+- `kado keybase note create-with-tags <note_name> <tag1,tag2,...>`: Creates a new note with tags.
+- `kado keybase note search-by-tag <tag>`: Searches for notes with a specific tag.
+
+### Security Benefits
+
+- **Enhanced Security**: Store sensitive information like API keys and tokens securely in Keybase.
+- **Version Control**: Keybase notes are version-controlled, allowing you to track changes to sensitive information.
+- **Easy Sharing**: Securely share notes with team members using Keybase's encryption.
+- **Tagging System**: Organize your notes with tags for easy searching and categorization.
+
+### Using Keybase Notes in Templates
+
+You can reference Keybase notes in your templates using the `{{keybase:note:note_name}}` syntax. This allows you to keep sensitive information like API keys and tokens secure while still being able to use them in your configurations.
+
+### Getting Started with Keybase Integration
+
+1. Ensure you have Keybase installed and configured on your system.
+2. Run `kado keybase link` to link your Keybase account with Kado.
+3. Create notes for sensitive information: `kado keybase note create <note_name>`
+4. Use note references in your bead definitions as shown in the example above.
+
+In above template example, `{{keybase:note:secret_token}}` will be replaced with the content of the corresponding Keybase notes during Kado execution.
+
 ## Usage
 
 ### Commands
 
 - `kado [file.yaml]`: Runs the default configuration and processing of beads. You may pass a specific YAML file to Kado. If no file is specified, Kado scans all YAML files in the current directory.
 - `kado set`: Applies the configuration and processes beads with the `set` flag.
 - `kado fmt [dir]`: Formats `.kd` files in the specified directory.
-- `kado ai`: Runs AI-based recommendations if enabled in the `~/.kdconfig` configuration.
+- `kado ai`: Runs AI-based recommendations if enabled.
 - `kado config`: Displays the current configuration and order of execution.
-- `kado -debug`: Runs Kado with debug output enabled, providing detailed information about the execution process.
+- `kado -debug`: Runs Kado with debug output enabled.
+- `kado keybase <command>`: Manages Keybase integration (link, create/list/view/share notes).
 
 ### Getting Started
 

VERSION

@@ -1 +1 @@
-0.1.0
+0.2.0

cluster.kd

@@ -14,4 +14,4 @@ bead "terraform" {
   enabled = true
   relay = opa
   relay_field = "source=git@github.com:janpreet/proxmox_terraform.git,path=terraform/policies/proxmox.rego,input=terraform/plan.json,package=data.terraform.allow"
-}
+}