diff --git a/.secrets.baseline b/.secrets.baseline index dad0a32c..7759d495 100644 --- a/.secrets.baseline +++ b/.secrets.baseline @@ -3,7 +3,7 @@ "files": "^.secrets.baseline$", "lines": null }, - "generated_at": "2024-07-02T16:36:02Z", + "generated_at": "2024-07-11T17:28:47Z", "plugins_used": [ { "name": "AWSKeyDetector" @@ -63,7 +63,7 @@ "hashed_secret": "64ab0c1d3edc1c8c166351207b840ac7b2a90523", "is_secret": false, "is_verified": false, - "line_number": 87, + "line_number": 59, "type": "Secret Keyword" } ], @@ -88,7 +88,7 @@ "hashed_secret": "ac0fedaac180de6bd70a97b711692a92dade479e", "is_secret": false, "is_verified": false, - "line_number": 94, + "line_number": 92, "type": "Secret Keyword" } ], diff --git a/README.md b/README.md index 364a65d7..4c7ac9b9 100644 --- a/README.md +++ b/README.md @@ -14,20 +14,9 @@ helm repo update helm upgrade --install gen3 gen3/gen3 -f ./values.yaml ``` -Assuming you already have the [prerequisites](./docs/PREREQUISITES.md) installed and configured, you can deploy Gen3 with the helm command. +For more information on how to deploy Gen3 with helm, please see the [Gen3 Example Deployment Guide](https://docs.gen3.org/docs/Deployment/Example%20Deployment) - -> **Warning** -> The default Helm chart configuration is not intended for production. The default chart creates a proof of concept (PoC) implementation where all Gen3 services are deployed in the cluster, including postgres and elasticsearch. For production deployments, you must follow the [Production/Cloud Native/Hybrid architecture](./docs/PRODUCTION.md) - - -For a production deployment, you should have strong working knowledge of Kubernetes. This method of deployment has different management, observability, and concepts than traditional deployments. - -In a production deployment: - -- The stateful components, like PostgreSQL or Elasticsearch, must run outside the cluster on PaaS or compute instances. This configuration is required to scale and reliably service the variety of workloads found in production Gen3 environments. - -- You should use Cloud PaaS for PostgreSQL, Elasticsearch, and object storage. +https://docs.gen3.org ## Configuration @@ -56,23 +45,6 @@ fence: ``` - -## Selective deployments -All gen3 services are sub-charts of the gen3 chart (which acts as an umbrella chart). - -For your specific installation of gen3, you may not require all our services. - - -To enable or disable a service you can use this pattern in your `values.yaml` - -```yaml -fence: - enabled: true - -wts: - enabled: false -``` - ## Gen3 Login Options Gen3 does not have any IDP, but can integrate with many. We will cover Google login here, but refer to the fence documentation for additional options. @@ -108,39 +80,6 @@ For `"Authorized redirect URIs"` add `https:///user/login/google/logi After configuration is complete, take note of the client ID that was created. You will need the client ID and client secret to complete the next steps. -# Production deployments -Please read [this](./docs/PRODUCTION.md) for more details on production deployments. - -NOTE: Gen3 helm charts are currently not used in production by CTDS, but we are aiming to do that soon and will have additional documentation on that. - -# Local Development - -For local development you must be connected to a kubernetes cluster. As referenced above in the section `Kubernetes cluster` we recommend using [Rancher Desktop](https://rancherdesktop.io/) as Kubernetes on your local machine, especially on M1 Mac's. You also get ingress and other benefits out of the box. - -For MacOS users, [Minikube](https://minikube.sigs.k8s.io/docs/start/) equipped with the ingress addon serves as a viable alternative to Rancher Desktop. On Linux, we've observed that using [Kind](https://kind.sigs.k8s.io/) with an NGINX ingress installed often provides a more seamless experience compared to both Rancher Desktop and Minikube. Essentially, Helm requires access to a Kubernetes cluster with ingress capabilities, facilitating the loading of the portal in your browser for an optimal development workflow. - -To install the NGINX ingress: -``` - helm repo add nginx-stable https://helm.nginx.com/stable - helm repo update - kubectl create ns nginx-ingress - helm install nginx-ingress nginx-stable/nginx-ingress --namespace nginx-ingress -``` - -> **Warning** -> If you are using Rancher Desktop you need to increase the vm.max_map_count as outlined [here](https://docs.rancherdesktop.io/how-to-guides/increasing-open-file-limit/) -> If you are using Minikube you will need to enabled the ingress addon as outlined [here](https://kubernetes.io/docs/tasks/access-application-cluster/ingress-minikube/) - -1. Clone the repository -2. Navigate to the `gen3-helm/helm/gen3` directory and run `helm dependency update` -3. Navigate to the back to the `gen3-helm` directory and create your values.yaml file. See the `TL;DR` section for a minimal example. -4. Run `helm upgrade --install gen3 ./helm/gen3 -f ./values.yaml` - -## Using Skaffold - -Skaffold is a tool for local development that can be used to automatically rebuild and redeploy your application when changes are detected. A minimal skaffold.yaml configuration file has been provided in the gen3-helm directory. Update the values of this file to match your needs. - -Follow the steps above, but instead of doing the helm upgrade --install step, use `skaffold dev` to start the development process. Skaffold will automatically build and deploy your application to your kubernetes cluster. # Troubleshooting diff --git a/docs/INGRESS.md b/docs/INGRESS.md index c51f74d6..cf97466a 100644 --- a/docs/INGRESS.md +++ b/docs/INGRESS.md @@ -1,7 +1,7 @@ # Ingress in Gen3 # Dev -if `global.dev` is set to true, a very basic ingress is created, that works out of the box with `traefik` service that is included in `Rancher-Desktop` +if `global.dev` is set to true, a very basic ingress is created. # AWS diff --git a/docs/PREREQUISITES.md b/docs/PREREQUISITES.md index bbdda80e..8f022d57 100644 --- a/docs/PREREQUISITES.md +++ b/docs/PREREQUISITES.md @@ -24,9 +24,7 @@ Before deploying the Gen3 application using Helm, ensure that the following prer ## Prerequisites ### Kubernetes cluster -Any kubernetes cluster _should_ work. We are testing with EKS, AKS, GKE and Rancher Desktop. - -It is suggested to use [Rancher Desktop](https://rancherdesktop.io/) as Kubernetes on your laptop, especially on M1 Mac's. You also get ingress and other benefits out of the box. +Any kubernetes cluster _should_ work. We are testing with EKS, AKS, GKE. ### Postgres diff --git a/docs/gen3_developer_environments.md b/docs/gen3_developer_environments.md deleted file mode 100644 index 81994149..00000000 --- a/docs/gen3_developer_environments.md +++ /dev/null @@ -1,199 +0,0 @@ -# Running Gen3 On A Laptop, For Devs - -Welcome to Gen3! - -If you're reading this, I assume you're a developer looking to get started working with Gen3. If so, this guide will help you get an instance of Gen3 up and running on your laptop, which will let you work on Gen3 from anywhere you can write code, no other setup needed. - - -# Kubernetes -Gen3 runs entirely on Kubernetes. Kubernetes is a container orchestrator, and you'll often see it referred to as k8s, or 'k', followed by the 8 letters in 'ubernete', followed by 's'. It is responsible for managing the lifecycle, storage, and networking for a collection of containers, which are packaged into discrete units called "pods". - -If you're not familiar with containers, they're a set of technologies that allow you to run code inside isolated environments on your machine. This provides benefits such as allowing you to manage "machnes" that only exist in software, instead of physical hardware, and isolating applications from each other, to prevent failures from affecting other applications or even all applications on your machine. These are the basic units software that Kubernetes starts up, stops, and schedules, in order to match a state that you define and provide it. - -Kubernetes is primarly meant to run on cloud services, and the big 3 (Amazon Web Services, Microsoft Azure, and Google Cloud) all have robust offerings. While it is designed to run primarily on the cloud, for the sake of developers' ability to work almost anywhere, there are versions (also called flavors) of Kubernetes designed primarily with the laptop or local desktop in mind. The next section will introduce you to the one we'll recommend you use, Rancher Desktop. However you can use another flavor if you think it works better, so long as you can follow the rest of these directions while using them. - -### Installing `kubectl` -Kubectl (I pronounce it cube-cuddle, but others say it differently) is an application that allows you to interface with and control a Kubernetes cluster. In this context, "cluster" simply refers to a group of machines, which can be a group of one, that work together to manage containers. - -It will be an important tool for you as you get more comfortable working with Kubernetes, so we'll install it now. It is a highly-configurable tool that you can install in a lot of ways, so rather than tell you how to do it, we'll let you pick the method that works best for your platform: [Windows](https://kubernetes.io/docs/tasks/tools/install-kubectl-windows) [Mac](https://kubernetes.io/docs/tasks/tools/install-kubectl-macos) [Linux](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux). - -Once you have kubectl installed, you can verify by running `kubectl`. The output should be a help guide. - -### Installing `helm` -Helm is a package manager for Kubernetes that makes it easy to install, upgrade, and manage applications on a Kubernetes cluster. It simplifies the process of installing and configuring complex applications by providing a set of pre-configured templates and options. In this guide, we will show you how to install the Helm command-line interface (CLI) on your machine. - -The installation steps for Helm are rather straightforward. If you're a Homebrew user on Mac, you can use the command `brew install helm` to get it on your machine. If not, head over to the Helm website [here](https://helm.sh/docs/intro/install/) and follow the instructions for your setup. You'll know you've set it up correctly if the output of running the command `helm list` looks like this: - -![image](images/succesfulHelmOutput.png) - -This means that Helm was able to connect to your Rancher k8s cluster, and will be able to install Gen3 in the next step. - - -### Kubernetes on your laptop -There are several ways to run Kubernetes on your laptop, depending on your needs and the resources available on your machine. Some of the most popular options include: - -**Minikube:** Minikube is a lightweight Kubernetes distribution that runs a single-node cluster on your laptop. It is easy to set up and is well suited for local development and testing. - -**Docker for Desktop:** Docker for Desktop includes built-in support for Kubernetes, allowing you to run a single-node cluster on your laptop using the Docker engine. This option is also easy to set up and is well suited for local development and testing. - -**k3s:** k3s is a lightweight Kubernetes distribution that is designed to run on resource-constrained environments. It is a great option for running Kubernetes on your laptop if you have limited resources or need to run multiple clusters. - -**Kind (Kubernetes in Docker):** Kind is a tool for running local Kubernetes clusters using Docker container “nodes”. It creates a cluster by starting multiple Docker containers on the local host. - -**Microk8s:** Microk8s is a fast and efficient Kubernetes distribution that is easy to install and run on a local machine. It uses snaps to package and distribute Kubernetes, making it a great option for users on Ubuntu and other Linux distributions. - -**Vagrant and Virtualbox:** Vagrant is a tool that enables you to create and configure lightweight, reproducible, and portable development environments. You can use it to create a virtual machine running Kubernetes on your laptop using Virtualbox. - -Each of these methods has its own advantages and disadvantages, so you should choose the one that best fits your needs. - -Another option to run Kubernetes on your laptop is **Rancher Desktop.** - -**Rancher Desktop** is an easy-to-use, all-in-one Kubernetes platform that runs on your local machine. It provides a simple and intuitive UI for managing your local Kubernetes cluster, and includes built-in support for Ingress, cert-manager, and other popular add-ons. Rancher Desktop also comes with a built-in Kubernetes dashboard and a set of tools for managing and monitoring your cluster. - -**Rancher Desktop** is our preferred way of running Kubernetes on a laptop, because it provides a user-friendly interface and comes with many pre-configured components, which make it easy to set up and manage your cluster. Additionally, it works well on M1 macbooks, which have new ARM-based processors. - - -### Installing Rancher Desktop -This guide is primarily written with Mac users in mind, but most Linux users should be able to install Rancher Desktop using their distro's package manager. If you're on Mac and using Homebrew, you can type `brew install --cask rancher`. If not, install directions for Linux, Mac, and Windows are available at [the Rancher website.](https://docs.rancherdesktop.io/getting-started/installation/#macOS) Once you have Rancher succesfully installed and the application opened, we can go over how to get your Kubernetes cluster ready for Gen3! - -### Configuring Rancher Desktop -![image](images/rancherReadyForSetup.png) - -Once you can see a blank screen like this, you are ready to begin. If you can't get a screen similar to this, without any warnings, reach out to a Gen3 resource, either the community, or the platform team if you work directly for us. Now, we're going to make a few small tweaks to help Kubernetes run better. - -Click on the gear icon in the top right of your window, then navigate to "virtual machine." These settings control the VM that Kubernetes is going to run on on your laptop, and so striking a the right balance between performance and resource usage is key. - -This guide was developed by people mostly using M1 Macbook Pros with 16GB of RAM and and 8 CPU cores. In a similar situation, this guide recommends allocating half of each (so 8GB of RAM/4 CPU cores) to allow you to run other applications while still deploying all of our services quickly. - -Once you've settled on a CPU and RAM allocation, click on the "Kubernetes" tab. Make sure that Kubernetes is enabled, and the version is set appropriately (if you're not sure, just leave it default). - -Now that you have these steps out of the way, in our next step, we'll install Helm onto our laptop. Helm is a tool for packaging Kubernetes services, much like a Linux package manager or Homebrew for Mac. This will allow us to more easily install Gen3 onto our laptops. - - - -### Installing Gen3 -The first step to installing Gen3 is adding the Gen3 Helm repository. This is just how we package up all the components that make up Gen3, and make them accessible to the public. - -The command to do this is: - -``` -helm repo add gen3 http://helm.gen3.org -helm repo update -``` - -Once you have your repo added, you can install it with the command - -``` -helm upgrade --install dev gen3/gen3 -``` - -If you want to provide overrides you can do so by passing in one, or several values.yaml files. F.ex if you want to pass in user.yaml and fence-config (NB! New format, check out sample files in [this](../sample-values/) folder) - -``` -helm upgrade --install dev gen3/gen3 -f values.yaml -f fence-config.yaml -f user.yaml -``` - - - -See example files: -- [values.yaml](../sample-values/values.yaml) -- [fence-config.yaml](../sample-values/fence-config.yaml) -- [user.yaml](../sample-values/user.yaml) - - -You can combine it all in a single file too if that's easier. - -This command calls out to the repository you created before, named `gen3`, and grabs an "umbrella chart" containing all the services needed to run Gen3. - -Confusingly, this umbrella chart is also called `gen3`, and these two parts combine to form the `gen3/gen3` in the command you see. - -The first `dev` in that command refers to the "release name," or what Helm will call the deployment of Gen3 on your laptop. If that command runs successfully, you will see an output like this: - -![image](helmSuccesfulGen3Install.png) - -### Accessing Gen3 -If everything went well with deploying you should now have an ingress resource, listening for the hostname you provided. - -``` -kubectl get ingress -``` - -If you used Rancher Desktop, and used localhost as your hostname, you should be able to access your application at `https://localhost/` - - -# Troubleshooting - -## Error: couldn't find key dbcreated in Secret default/\*-dbcreds - -This is by design, it’s waiting for your dbcreate job(s) to finish which signals that the services can start. - -## Local Dev Linux (Ubuntu) / Rancher Desktop Problems: - -If you've followed the instructions, see services running, but can't go to `https://localhost` to see portal... keep reading. - -If you see an Apache webpage at `localhost` (installed by default in Ubuntu), you have to kill the Apache service. - -``` -sudo systemctl stop apache2 -``` - -If things still aren't working, reference [this](https://github.com/rancher-sandbox/rancher-desktop/issues/1668), which suggests trying this: - -``` -sudo sysctl net.ipv4.ip_unprivileged_port_start -sudo sysctl net.ipv4.ip_unprivileged_port_start=80 -``` - -Then restart Rancher and try again. - -If things still aren't working, double check your `values.yaml` and make sure you don't have anything in the `global` block (if you're doing local development). Just let it use the defaults (so don't put anything in `values.yaml` for `global`). - - -## Elasticsearch error: - -When you deploy the elasticsearch chart to Rancher Desktop you may see this error and elasticsearch failing to start: - -``` -ERROR: [1] bootstrap checks failed -[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] -``` - -To fix it do the following: - -Get a shell to the VM running k8s/docker in rancher -``` -rdctl shell -``` - -Run this: - -``` -sudo sysctl -w vm.max_map_count=262144 -``` - -If you want to set this permanently, you need to edit `/etc/sysctl.conf` and set `vm.max_map_count` to `262144` . Remember to do this inside the `rdctl shell` too. - -When the host reboots, you can verify that the setting is still correct by running - -``` -sysctl vm.max_map_count -``` - - -# Cool kubernetes tools -Another way to interact with and manage a Kubernetes cluster is by using command-line tools, such as k9s. - -K9s is a terminal-based tool that provides a simple and intuitive UI for interacting with your local Kubernetes cluster. It includes features such as pod management, resource monitoring, and log viewing. k9s also provides a live view of your cluster, making it easy to identify and troubleshoot issues. - -Other similar tools for developers include: - -- Stern: a multi-pod and container log tailing for Kubernetes -- Kube-ps1: a Kubernetes prompt for bash and zsh -- Kube-shell: An integrated shell for working with the Kubernetes CLI -- Skaffold: a command line tool that facilitates continuous development for Kubernetes applications. - - -These command-line tools can be a great option for developers who prefer to work in the terminal and want a more streamlined and efficient way to interact with their cluster. They are lightweight, easy to install and have a small footprint. They provide a simple and efficient way to manage and monitor your cluster. - - -