Skip to content

Latest commit

 

History

History
221 lines (184 loc) · 6.4 KB

configuration.md

File metadata and controls

221 lines (184 loc) · 6.4 KB

k0s configuration

Control plane

k0s Control plane can be configured via a YAML config file. By default k0s server command reads a file called k0s.yaml but can be told to read any yaml file via --config option.

An example config file with defaults generated by the k0s default-config command:

apiVersion: ""
kind: ""
metadata: null
spec:
  api:
    address: 192.168.68.106
    sans:
    - 192.168.68.106
    - 192.168.68.106
    extraArgs: {}
  controllerManager:
    extraArgs: {}
  scheduler:
    extraArgs: {}
  storage:
    type: etcd
    kine: null
    etcd:
      peerAddress: 192.168.68.106
  network:
    podCIDR: 10.244.0.0/16
    serviceCIDR: 10.96.0.0/12
    provider: calico
    calico:
      mode: vxlan
      vxlanPort: 4789
      vxlanVNI: 4096
      mtu: 1450
      wireguard: false
  podSecurityPolicy:
    defaultPolicy: 00-k0s-privileged
  workerProfiles: []
images:
  konnectivity:
    image: us.gcr.io/k8s-artifacts-prod/kas-network-proxy/proxy-agent
    version: v0.0.13
  metricsserver:
    image: gcr.io/k8s-staging-metrics-server/metrics-server
    version: v0.3.7
  kubeproxy:
    image: k8s.gcr.io/kube-proxy
    version: v1.19.0
  coredns:
    image: docker.io/coredns/coredns
    version: 1.7.0
  calico:
    cni:
      image: calico/cni
      version: v3.16.2
    flexvolume:
      image: calico/pod2daemon-flexvol
      version: v3.16.2
    node:
      image: calico/node
      version: v3.16.2
    kubecontrollers:
      image: calico/kube-controllers
      version: v3.16.2
  repository: ""
telemetry:
  interval: 10m0s
  enabled: true
helm:
  repositories:
  - name: stable
    url: https://charts.helm.sh/stable
  - name: prometheus-community
    url: https://prometheus-community.github.io/helm-charts
  charts:
  - name: prometheus-stack
    chartname: prometheus-community/prometheus
    version: "11.16.8"
    values: |2
        values: "for overriding"
    namespace: default

spec.storage

  • type: Type of the data store, either etcd or kine.
  • etcd.peerAddress: Nodes address to be used for etcd cluster peering.
  • kine.dataSource: kine datasource URL.

Using type etcd will make k0s to create and manage an elastic etcd cluster within the controller nodes.

spec.api

  • address: The local address to bind API on. Also used as one of the addresses pushed on the k0s create service certificate on the API. Defaults to first non-local address found on the node.
  • sans: List of additional addresses to push to API servers serving certificate

spec.network

  • provider: Network provider, either calico or custom. In case of custom user can push any network provider.
  • podCIDR: Pod network CIDR to be used in the cluster
  • serviceCIDR: Network CIDR to be used for cluster VIP services.

spec.network.calico

  • mode: vxlan (default) or ipip
  • vxlanPort: The UDP port to use for VXLAN (default 4789)
  • vxlanVNI: The virtual network ID to use for VXLAN. (default: 4096)
  • mtu: MTU to use for overlay network (default 1450)
  • wireguard: enable wireguard based encryption (default false). Your host system must be wireguard ready. See https://docs.projectcalico.org/security/encrypt-cluster-pod-traffic for details.

spec.podSecurityPolicy

Configures the default psp to be set. k0s creates two PSPs out of box:

  • 00-k0s-privileged (default): no restrictions, always also used for Kubernetes/k0s level system pods
  • 99-k0s-restricted: no host namespaces or root users allowed, no bind mounts from host

As a user you can of course create any supplemental PSPs and bind them to users / access accounts as you need.

spec.workerProfiles

Array of spec.workerProfiles.workerProfile Each element has following properties:

  • name: string, name, used as profile selector for the worker process
  • values: mapping object

For each profile the control plane will create separate ConfigMap with kubelet-config yaml. Based on the --profile argument given to the k0s worker the corresponding ConfigMap would be used to extract kubelet-config.yaml from. values are recursively merged with default kubelet-config.yaml

There are a few fields that cannot be overridden:

  • clusterDNS
  • clusterDomain
  • apiVersion
  • kind

Example:

  workerProfiles:
    - name: custom-role
      values:
         key: value
         mapping:
             innerKey: innerValue

images

Each node under the images key has the same structure

images:
    konnectivity:
      image: calico/kube-controllers
      version: v3.16.2

Following keys are avaiable

images.konnectivity

images.metricsserver

images.kubeproxy

images.coredns

images.calico.cni

images.calico.flexvolume

images.calico.node

images.calico.kubecontrollers

images.repository

If images.repository is set and not empty, every image name will be prefixed with the value of images.repository

Example

images:
  repository: "my.own.repo"
    konnectivity:
      image: calico/kube-controllers
      version: v3.16.2

In the runtime the image name will be calculated as my.own.repo/calico/kube-controllers:v3.16.2

Telemetry

To build better end user experience we collect and send telemetry data from clusters. It is enabled by default and can be disabled by settings corresponding option as false The default interval is 2 minutes, any valid value for time.Duration string representation can be used as a value. Example

telemetry:
  interval: 2m0s
  enabled: true

Addons

It is possible to bundle 3rd party helm charts with your cluster.

helm:
  repositories:
  - name: stable
    url: https://charts.helm.sh/stable
  - name: prometheus-community
    url: https://prometheus-community.github.io/helm-charts
  charts:
  - name: prometheus-stack
    chartname: prometheus-community/prometheus
    version: "11.16.8"
    values: |2
        values: "for overriding"
    namespace: default

Configuring multi-node controlplane

When configuring an elastic/HA controlplane one must use same configuration options on each node for the cluster level options. Following options need to match on each node, otherwise the control plane component will end up in very unknown states:

  • network
  • storage: Needless to say, one cannot create a clustered controlplane with each node only storing data locally on SQLite.