Helm repo, or OCI repo, that's the question

[bbaassssiiee]

In an offline production environment, the Kubernetes cluster cannot access GitHub/internet, only private repositories (hosted in Artifactory/Sonatype Nexus). What steps do we need to take to enable offline installation? I'd like to use Helm exclusively. Do I need to create a traditional Helm repo for the index.yaml, or would a OCI repo suffice?

I can populate the OCI repo with the Ansible below, but that does not create the index.yaml:

- name: Ensure KubeBlocks cache
  ansible.builtin.file:
    path: "{{ kubeblocks_cache }}"
    state: directory
    mode: '0755'

- name: Pull KubeSpray helm charts
  kubernetes.core.helm_pull:
    chart_ref: "{{ item.addon_name }}"
    chart_version: "{{ item.version }}"
    repo_url: "{{ helm_upstream_repo_url }}"
    untar_chart: false
    destination: "{{ kubeblocks_cache }}"
  when:
    - item.state == "present"
  loop: "{{ kubeblocks_add_ons }}"

- name: Registry login for helm
  environment:
    REGISTRY_PASS: "{{ registry_pass }}"
  ansible.builtin.command:
    cmd: 'helm registry login "{{ registry_host }}" --username "{{ registry_user }}" --password $REGISTRY_PASS'

- name: Push helm charts
  ansible.builtin.command:
    cmd: "helm push {{ kubeblocks_cache }}/{{ item.addon_name }}-{{ item.version }}.tgz oci://{{ registry_host }}/helm-apecloud/"
  when:
    - item.state == "present"
  loop: "{{ kubeblocks_add_ons }}"

[JashBook]

Thank you for reaching out with your detailed question. To address your requirements for an offline Kubernetes environment using Helm exclusively, here are the steps and considerations you need to take into account:

1. Since you are using an OCI repository, a traditional Helm repo (which includes the index.yaml file) is not necessary. The OCI repository manages Helm charts directly through the OCI protocol and does not require an additional index.yaml file.
2. If you need to deploy KubeBlocks using Helm in an offline production environment, push the KubeBlocks Helm charts to your private OCI repository using the helm push command, as shown in your Ansible script.
3. Container images referenced in charts must also be synced to a private registry (e.g., Harbor). Override image paths during installation using --set:

helm install my-app oci://... --set image.registry=private-registry.example.com

The following are images related to deploying KubeBlocks v0.9.2:

# KubeBlocks
docker.io/apecloud/kubeblocks:0.9.2
docker.io/apecloud/kubeblocks-dataprotection:0.9.2
docker.io/apecloud/kubeblocks-datascript:0.9.2
docker.io/apecloud/kubeblocks-tools:0.9.2
docker.io/apecloud/kubeblocks-charts:0.9.2

4. With Charts and Images prepared, you can use the following commands to deploy KubeBlocks:

helm install kubeblocks oci://registry.example.com/helm-oci/kubeblocks --version v0.9.2 --namespace kb-system --create-namespace --set image.registry=private-registry.example.com --set dataProtection.image.registry=private-registry.example.com --set addonChartsImage.registry=private-registry.example.com

5. The following are images related to create cluster (mysql/postgresql/redis/mongodb)

# mysql
docker.io/apecloud/mysql:5.7.44
docker.io/apecloud/mysql:8.0.30
docker.io/apecloud/mysql:8.0.31
docker.io/apecloud/mysql:8.0.32
docker.io/apecloud/mysql:8.0.33
docker.io/apecloud/mysql:8.0.34
docker.io/apecloud/mysql:8.0.35
docker.io/apecloud/mysql:8.0.36
docker.io/apecloud/mysql:8.0.37
docker.io/apecloud/mysql:8.0.38
docker.io/apecloud/mysql:8.0.39
docker.io/apecloud/mysql:8.4.0
docker.io/apecloud/mysql:8.4.1
docker.io/apecloud/mysql:8.4.2
docker.io/apecloud/agamotto:0.1.2-beta.1
docker.io/apecloud/jemalloc:5.3.0
docker.io/apecloud/mysql_audit_log:5.7.44
docker.io/apecloud/mysql_audit_log:8.0.33
docker.io/apecloud/percona-xtrabackup:2.4
docker.io/apecloud/percona-xtrabackup:8.0
docker.io/apecloud/percona-xtrabackup:8.4
docker.io/apecloud/syncer:0.3.9
docker.io/apecloud/xtrabackup:2.4
docker.io/apecloud/xtrabackup:8.0
docker.io/apecloud/xtrabackup:8.4
# postgresql
docker.io/apecloud/pgbouncer:1.19.0
docker.io/apecloud/postgres-exporter:v0.15.0
docker.io/apecloud/spilo:12.15.0-pgvector-v0.6.1
docker.io/apecloud/spilo:12.14.0-pgvector-v0.6.1
docker.io/apecloud/spilo:14.8.0-pgvector-v0.6.1
docker.io/apecloud/spilo:12.14.1-pgvector-v0.6.1
docker.io/apecloud/spilo:14.7.2-pgvector-v0.6.1
docker.io/apecloud/spilo:15.7.0
docker.io/apecloud/spilo:16.4.0
docker.io/apecloud/wal-g:postgres-1.0
# redis
docker.io/apecloud/agamotto:0.1.2-beta.1
docker.io/apecloud/ape-dts:0.1.17
docker.io/apecloud/busybox:1.36
docker.io/apecloud/twemproxy:0.5.0
docker.io/apecloud/redis-stack-server:7.0.6-RC4
docker.io/apecloud/redis-stack-server:7.0.6-RC8
docker.io/apecloud/redis-stack-server:7.2.0-v10
# mongodb
docker.io/apecloud/agamotto:0.1.2-beta.1
docker.io/apecloud/mongo:4.0.28
docker.io/apecloud/mongo:4.2
docker.io/apecloud/mongo:4.2.24
docker.io/apecloud/mongo:4.4
docker.io/apecloud/mongo:4.4.29
docker.io/apecloud/mongo:5.0.14
docker.io/apecloud/mongo:5.0.28
docker.io/apecloud/mongo:6.0
docker.io/apecloud/mongo:6.0.16
docker.io/apecloud/mongo:7.0.12
docker.io/apecloud/syncer:0.3.7

6. With Addon Charts and Images prepared, you can use the following commands to create cluster:

# mysql
helm install mysql-cluster oci://registry.example.com/helm-oci/mysql-cluster --version 0.9.1
# postgresql
helm install pg-cluster oci://registry.example.com/helm-oci/postgresql-cluster --version 0.9.0
# redis
helm install redis-cluster oci://registry.example.com/helm-oci/redis-cluster --version 0.9.0
# mongodb
helm install mongo-cluster oci://registry.example.com/helm-oci/mongodb-cluster --version 0.9.1

[bbaassssiiee]

I noticed busybox:latest was missing from this list.

[bbaassssiiee]

7. For other charts you can derive the images used with

helm template stdout $CHART | grep image: | sed 's/image://;s/ //g;s/^-//;/IMAGE/d' | sort -u

[bbaassssiiee]

I still get ImagePullBackOff when I do all the suggested steps, have every chart and image in our registry_host, and deploy with:

- name: Manage the KubeBlocks chart
  kubernetes.core.helm:
    binary_path: "{{ helm_binary }}"
    name: kubeblocks
    chart_ref: "{{ kubeblocks_repo }}/kubeblocks-{{ kubeblocks_version }}.tgz"
    release_namespace: kb-system
    create_namespace: true
    reuse_values: true
    values:
      image.registry: "{{ registry_host }}"
      dataProtection.image.registry: "{{ registry_host }}"
      addonChartsImage.registry: "{{ registry_host }}"
    state: "{{ desired_state }}"
    wait: true

[JashBook]

I still get ImagePullBackOff when I do all the suggested steps, have every chart and image in our registry_host, and deploy with:

- name: Manage the KubeBlocks chart
  kubernetes.core.helm:
    binary_path: "{{ helm_binary }}"
    name: kubeblocks
    chart_ref: "{{ kubeblocks_repo }}/kubeblocks-{{ kubeblocks_version }}.tgz"
    release_namespace: kb-system
    create_namespace: true
    reuse_values: true
    values:
      image.registry: "{{ registry_host }}"
      dataProtection.image.registry: "{{ registry_host }}"
      addonChartsImage.registry: "{{ registry_host }}"
    state: "{{ desired_state }}"
    wait: true

Sorry, missed a step.
8. After KubeBlocks is deployed, the addon that is enabled by default needs to be manually modified again --set image.registry=private-registry.example.com or --set images.registry=private-registry.example.com.
Other addons that need to be enabled later also need to set these two values.
You can use the following commands to set addon values

helm upgrade -n kb-system kb-addon-<addon-name> oci://registry.example.com/helm-oci/<addon-name> --set image.registry=private-registry.example.com --set images.registry=private-registry.example.com

[bbaassssiiee]

helm install kubeblocks oci://registry.example.com/helm-oci/kubeblocks --version v0.9.2 --namespace kb-system --create-namespace --set image.registry=private-registry.example.com --set dataProtection.image.registry=private-registry.example.com --set addonChartsImage.registry=private-registry.example.com

[JashBook]

helm install kubeblocks oci://registry.example.com/helm-oci/kubeblocks --version v0.9.2 --namespace kb-system --create-namespace --set image.registry=private-registry.example.com --set dataProtection.image.registry=private-registry.example.com --set addonChartsImage.registry=private-registry.example.com

To prevent the KubeBlocks Helm Chart from being too large, we have removed CRD from the Helm Chart since version 0.8. CRD needs to be installed before deployment. For offline environments, please save them in an accessible address

kubectl replace -f https://github.com/apecloud/kubeblocks/releases/download/v0.9.2/kubeblocks_crds.yaml

[bbaassssiiee]

Thanks for the quick reply. I noticed my registry is incomplete at the moment.

[bbaassssiiee]

Note that for instance the snapshot-controller deviates from this pattern and needs --set image.repository

[bbaassssiiee]

I published a solution with private OCI registry
#8922