CRoSSFITs Infrastructure

This repository contains the configuration to build and deploy the machine images in use in my homelab.

Packer

infra/packer
├── rocky-9
│   ├── files
│   └── http
└── ubuntu-2204
    ├── credentials.auto.pkrvars.hcl
    ├── files
    │   └── 99-pve.cfg
    ├── http
    │   ├── meta-data
    │   ├── user-data
    │   └── vendor-data
    └── pve-ubuntu-2204-amd64-qemu.pkr.hcl

ubuntu-2204/ contains the pve-ubuntu-2204-amd64-qemu.pkr.hcl file which defines the machine image for an Ubuntu Jammy (LTS 22.04) VM.

ubuntu-2204/ also contains the credentials.auto.pkvars.hcl file (not tracked in git) which contains credentials to authenticate to the PVE cluster.

ubuntu-2204/files contain files used during provisioning, namely cloud-init datasource configuration.

ubuntu-2204/http serves as the web root for packer to perform an unintended install of an Ubuntu Jammy machine image utilizing cloud-init.

rocky-9 is a work in progress.

Terraform

infra/terraform
├── credentials.auto.tfvars
├── data.tf
├── main.tf
├── providers.tf
├── pve-ubuntu-2204-amd64-qemu.manifest.json
├── templates
│   ├── network-data-rke.tftpl
│   ├── network-data.tftpl
│   ├── user-data-rke.tftpl
│   └── user-data.tftpl
└── variables.tf

credentials.auto.tfvars contains the credentials to authenticate to the PVE cluster (not tracked in git).

data.tf and providers.tf contain Terraform setup information.

main.tf is the resource definitions of the VMs present in my homelab infrastructure.

pve-ubuntu-2204-amd64-qemu.manifest.json is the template manifest that is generated by Packer.

templates/network-data*.tftpl and templates/user-data*.tftpl contain cloud-init customization data to further customize the deployed images as needed.

Kubernetes

infra/kubernetes
├── rke
│   ├── cert-manager-issuer.yml
│   ├── cluster.rkestate
│   ├── cluster.yml
│   ├── external-dns.yml
│   ├── hello-world.yml
│   ├── kube_config_cluster.yml
│   └── metallb.yml
└── services
    └── dashboard.yml

rke/cluster.yml is the RKE cluster definition. Running rke up --config cluster.yml will create the Kubernetes cluster with with the following services.

• MetalLB in L2 advertisement mode.
• ingress-nginx The default installation in RKE is disabled to expose the the ingress-nginx-controller service as type: LoadBalancer via MetalLB.
• cert-manager with an ACME ClusterIssuer to generate certificates for deployed services.
• ExternalDNS configured for RFC2136 for service discovery.
• Rancher Hello World to ensure that the above are deployed successfully.

Follow-on actions

Install Rook and connect it to the back-end (external) Ceph cluster. I am using Ceph to support my PVE cluster, Rook allows me to run kubernetes workloads and have durable storage directly on the underlying storage pools.

Set the default storage class to either ceph-rbd or cephfs

kubectl get storageclass
kubectl patch storageclass cephfs -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Install CNPG for database workloads.