Kubernetes Support Bundle Kit Design Target (Phase 1)

Problem

1. It's not easy to debug a Kubernetes native application. If you have collected the YAML files and logs for the pod, since it's natural of microservice, you have to look into different YAML and logs to find the needed information. It's hard to navigate through the YAML files if you only treat them as text files.
2. Many Kubernetes native projects (like Longhorn and Harvester) already have a built-in support bundle generation mechanism, which can help with troubleshooting. But many code are duplicated across different projects.
3. In addition to debug Kubernetes native applications, there is a need to help to debug Kubernetes itself, along with system-level logs. The Rancher support team has a script to collect Rancher/RKE/K3S related logs, but it's separated from the applications running on Kubernetes.

Goal

1. Create a unified support bundle generator for Kubernetes and Kubernetes native applications.
   1. The generator will collect the YAMLs and logs from user-specified namespaces, as well as logs from system-level service on the nodes from all the Kubernetes nodes.
   2. The generator can be run in two modes:
      1. As one-time progress directly from a URL with the configuration
      2. As a server to can communicate with a Kubernetes native application to
         1. Initiate the collecting
         2. Report collecting progress
         3. Provide a way to download the support bundle (most likely from the application UI).
   3. The format of the support bundle will be a ZIP file.
   4. Integration goal
      1. The generator should replace the current support bundle collection mechanism in Longhorn, Harvester. It should also replace the Rancher support bundle collecting script above.
2. Create a support bundle analyzer for the support bundle generated above.
   1. User should be able to import the support bundle directly into the analyzer.
   2. Analyzer should be able to generate a mocked Kubernetes environment using the information in the support bundle.
      1. User should be able to use kubectl or k9s to navigate the environment, as it's in the environment where the support bundle was collected.
      2. For non-Kubernetes log/information, the analyzer will provide a way for the user to easily navigate through them.
         1. Probably as a mocked Kubernetes resource inside the mocked Kubernetes environment.
   3. The generated support bundle should be useful and correct by itself.
      1. User should still able to inspect the support bundle as they wish when need. They don't have to import it into the analyzer.

Known technical challenges

1. How to collect all the resources within a namespace
   • Might need to get the list of available API resources in the cluster before trying to collect the YAML.
2. The correctness of the object from imported YAML
   • Imported YAMLs will result in a refresh creationTimestamp for the object.
3. How to hook up to the kubectl log command for the related logs.

References

Longhorn support bundle: https://github.com/longhorn/longhorn-manager/blob/27c93ca97bf6966c7a946ef223366f3ac6e70d62/manager/misc.go#L112

Harvester support bundle: this repo

KCP: https://github.com/kcp-dev/kcp