Autoscale OKE Node Pools with Agones Game Servers and Fleets (#443)

* adding OKE with Agones * fixing code blocks * testing markdown * fixing readme * fix readme * linting * linting * Update 4-create-agones-system.md adding -v to nc so user sees a success on the connect --------- Co-authored-by: nogbit <marcellus.miles@oracle.com>
oracle-livelabs · Jan 9, 2025 · e143ce5 · e143ce5
1 parent e57d2ba
commit e143ce5
Show file tree

Hide file tree

Showing 12 changed files with 868 additions and 0 deletions.
diff --git a/oke-with-agones/0-workshop-introduction/0-workshop-introduction.md b/oke-with-agones/0-workshop-introduction/0-workshop-introduction.md
@@ -0,0 +1,55 @@
+# Introduction
+
+## **What is OKE and Agones?**
+
+### OKE Intro
+
+Oracle Kubernetes Engine (OKE) is the runtime within OCI for the running and operations of enterprise-grade Kubernetes at scale. You can easily deploy and manage resource-intensive workloads such as dedicated game servers with automatic scaling, patching, and upgrades.
+
+### Agones Intro
+
+Agones is an open source platform, for deploying, hosting, scaling, and orchestrating dedicated game servers for large scale multiplayer games, built on top of the industry standard, distributed system platform Kubernetes.
+
+Agones replaces bespoke or proprietary cluster management and game server scaling solutions with an open source solution that can be utilized and communally developed - so that you can focus on the important aspects of building a multiplayer game, rather than developing the infrastructure to support it.
+
+### Workshop Lab Objectives
+
+* Ensure you have installed the prerequisites
+* Create the OCI infrastructure
+* Setup OKE Autoscaling
+* Setup Agones system pods with Helm
+* Deploy an Agones Fleet (dedicated game servers)
+* Scale a Agones Fleet and OKE nodes
+* Teardown
+
+### Labs
+
+| Module | Est. Time |
+| ------------- | :-----------: |
+| [Workshop Introduction](?lab=0-workshop-introduction) | 5 minutes |
+| [Get Started](?lab=1-get-started) | 15 minutes |
+| [Creating OCI Resources With Terraform](?lab=2-create-infrastructure-with-terraform) | 30 minutes |
+| [Installing the OKE Autoscaler Addon](?lab=3-install-oke-autoscaler-addon) | 20 minutes |
+| [Create the Agones System Pods with Helm](?lab=4-create-agones-system) | 15 minutes |
+| [Deploy an Agones Fleet and Autoscale OKE Nodes](?lab=5-create-scale-agones-fleet) | 25 minutes |
+| [Teardown](?lab=6-teardown) | 10 minutes |
+
+Total estimated time: 120 minutes
+
+## Task 1: Begin The Labs
+
+Use the left navigation on this page to begin the labs in this workshop.
+
+You may now **proceed to the next lab**
+
+## Learn More - *Useful Links*
+
+- [Kubernetes](https://kubernetes.io/)
+- [OKE](https://www.oracle.com/cloud/cloud-native/kubernetes-engine/)
+- [OKE Terraform Module](https://github.com/oracle-terraform-modules/terraform-oci-oke)
+- [Agones](https://agones.dev/site/docs/)
+
+## **Acknowledgements**
+
+ - **Author** - Marcellus Miles, Master Cloud Architect
+ - **Last Updated By/Date** - Marcellus Miles, Dec 2024
diff --git a/oke-with-agones/1-get-started/1-get-started.md b/oke-with-agones/1-get-started/1-get-started.md
@@ -0,0 +1,89 @@
+# Get Started
+
+In this lab you will install the necessary components for this workshop.
+
+## Introduction
+
+In order to complete this workshop you will need to have the necessary tooling to connect to and deploy OCI resources.
+
+You will be using Terraform to deploy to OCI and will also need the OCI CLI.
+
+Estimated Time: 15 minutes
+
+### Objectives
+
+In this lab, you will:
+ - Install the OCI CLI
+ - Install Terraform
+ - Download the Terraform files
+ - Initialize Terraform
+
+### Prerequisites
+
+Please ensure you have the following before continuing
+
+ - An OCI Tenancy
+ - A Shell ([OCI Cloud Shell](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/cloudshellintro.htm), Linux, MacOS, Windows with WSL)
+ - A user in a group with Tenancy Admin and downloaded API Key
+
+## Task 1: Install the OCI CLI
+
+Install the OCI CLI
+
+1. Make sure you have the policy for Tenancy admin. This is required because the Terraform OKE module creates a dynamic group policy, other than that everything gets created in a OCI Compartment you specify in the `terraform.tfvars` file.
+
+1. Follow the [install steps from Oracle](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm)
+
+2. Make sure you followed the steps above and are fully setup with API Keys (this will be the case if you did `oci setup config`) for the user of the above mentioned Tenancy admin.
+
+## Task 2: Install Terraform
+
+Complete the [install steps form Hashicorp](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)
+
+## Task 3: Download the Terraform Files
+
+Download the Terraform files
+
+1. Create a directory called `infrastructure` in your system
+
+    ````shell
+    <copy>
+    mkdir infrastructure
+    cd infrastructure
+    </copy>
+    ````
+
+2. Download the terraform files from [terraform.tar.gz](./files/terraform.tar.gz) to `infrastructure`.
+
+3. Untar the downloaded file
+
+    ````shell
+    <copy>
+    tar -xvzf terraform.tar.gz
+    </copy>
+    ````
+
+3. From within the infrastructure folder, initialize the Terraform
+
+    ````shell
+    <copy>
+    terraform init
+    </copy>
+    ````
+
+You may now **proceed to the next lab**
+
+## Learn More - *Useful Links*
+
+- [Kubernetes](https://kubernetes.io/)
+- [OKE](https://www.oracle.com/cloud/cloud-native/kubernetes-engine/)
+- [OKE Terraform Module](https://oracle-terraform-modules.github.io/terraform-oci-oke/)
+
+## **Summary**
+
+You have now initialized the dependencies for this Workshop.
+
+## **Acknowledgements**
+
+ - **Author** - Marcellus Miles, Master Cloud Architect
+ - **Last Updated By/Date** - Marcellus Miles, Dec 2024
diff --git a/oke-with-agones/1-get-started/files/terraform.tar.gz b/oke-with-agones/1-get-started/files/terraform.tar.gz
diff --git a/...-create-infrastructure-with-terraform/2-create-infrastructure-with-terraform.md b/...-create-infrastructure-with-terraform/2-create-infrastructure-with-terraform.md
@@ -0,0 +1,91 @@
+# Create OCI Resources With Terraform
+
+In this lab you will create the OCI Network, Bastion, Operator and OKE cluster using Terraform and the OKE Terraform module.  It's important to read through the [OKE Module documentation](https://oracle-terraform-modules.github.io/terraform-oci-oke/) as there are numerous options that can apply to your specific OCI deployment.
+
+## Introduction
+
+This Terraform deployment creates the following resources
+
+- Private OKE Control plane
+- Private Operator (with kubectl installed)
+- Public bastion (for SSH tunneling to Operator)
+- Three node pools (one public for game servers, and two private for the Autoscaler and Agones system pods respectively)
+- Security Group Rules for UDP access (game server to game client connectivity)
+- VCN Logs for logging traffic
+
+Estimated Time: 30 minutes
+
+### Objectives
+
+In this lab, you will:
+ - Configure your Terraform variables
+ - Run a Terraform plan and apply changes
+
+### Prerequisites
+
+ - Completed Lab 1 which walked through sourcing the Terraform files
+
+## Task 1: Update Terraform Variables
+
+Customize the infrastructure to fit your tenancy and compartment.
+
+1. You will want to edit `infrastructre/terraform.tfvars` with relevant information to match your account OCID's and API Keys.  For a full description of each variable in that file refer to `infrasctucture/variables.tf`.
+
+2. You can (optional) edit `infrastructure/module.tf` to tweak the OKE settings as needed for your deployment.  The current settings will work as is for the purpose of this lab. Its a good idea to have a look at these settings since the OKE Terraform module does a lot and is very customizable.
+
+## Task 2: Create a Terraform Plan
+
+Get a Terraform plan and check the output of that plan to make sure its what you expect.  After you validate the plan you can move onto the next task.
+
+````shell
+<copy>
+cd infrastructure
+terraform plan
+</copy>
+````
+
+## Task 3: Apply the Terraform Plan
+
+You can now apply the plan and the infrastructure will get created. Wait some time for the apply to complete.
+
+````shell
+<copy>
+terraform apply
+</copy>
+````
+
+## Task 4: Connect to the bastion
+
+Connect to the Bastion to ensure you have access to the OKE control plane.
+
+1. Get the terraform output. It will give an example command that you can use to SSH to the bastion and jump to the operator.
+
+    ````shell
+    <copy>
+    terraform output
+    </copy>
+    ````
+
+2. The operator has kubectl installed with connectivity to the OKE control plane.  **An example of that output is below**. You should test this command and make sure it works before proceeding to the next section of this workshop.
+
+    ````shell
+    <copy>
+    ssh -J opc@<bastion public IP> opc@<operator private ip>
+    </copy>
+    ````
+
+You may now **proceed to the next lab**
+
+## **Summary**
+
+You have now deployed the necessary infrastructure and connected to the Bastion and jumped to the Operator.  You are ready to begin installing the autoscaler and more.
+
+## Learn More - *Useful Links*
+
+- [OKE Terraform Module](https://oracle-terraform-modules.github.io/terraform-oci-oke/)
+- [Terraform Variables](https://developer.hashicorp.com/terraform/language/values/variables)
+
+## **Acknowledgements**
+
+ - **Author** - Marcellus Miles, Master Cloud Architect
+ - **Last Updated By/Date** - Marcellus Miles, Dec 2024
diff --git a/oke-with-agones/3-install-oke-autoscaler-addon/3-install-oke-autoscaler-addon.md b/oke-with-agones/3-install-oke-autoscaler-addon/3-install-oke-autoscaler-addon.md
@@ -0,0 +1,124 @@
+# Install the OKE Autoscaler Addon
+
+In this lab you will install and verify the OKE Cluster Autoscaler Add-on.
+
+## Introduction
+
+The OKE Cluster Autoscaler Add-on is what is used to watch and manage specified node pools that should be autoscaled.  The node pool you will configure this add on for is the pool that will be running the Agones fleet in future labs of this workshop.
+
+The add-on itself is installed into its own node pool to isolate it from its own scaling events.
+
+
+Estimated Time: 20 minutes
+
+### Objectives
+
+In this lab, you will:
+ - Verify the installation of the Autoscaler
+ - Create a config file for the Autoscaler Add-on
+ - Install the Autoscaler Add-on
+
+### Prerequisites
+
+ - Completed Lab 2 which walked through deploying the infrastructure
+
+## Task 1: Verify if the Autoscaler is already installed
+
+Depending on the OKE Terraform module used and your connectivity when creating the infrastructure the OKE Terraform may have installed the addon for you.  Its configured to do so, but does not always work in some scenarios.
+
+You should verify the current state of addons to see if the autoscaler was installed by following the steps below.
+
+1. SSH to your Operater using the output from `terraform output`, example below.
+
+    ```bash
+    <copy>
+    ssh -J opc@<bastion public IP> opc@<operator private ip>
+    </copy>
+    ```
+
+2. Get the OCID of your cluster by running this command and looking for the `id` (which is the OCID) of the cluster you just created.  This can also be obtained from the web console.
+
+    ````shell
+    <copy>
+    oci ce cluster list -c <OCID of Compartment you used in terraform.tfvars>
+    </copy>
+    ````
+
+3. Get the Addons installed on your cluster
+
+    ````shell
+    <copy>
+    oci ce cluster list-addons --cluster-id <OCID of your OKE Cluster from previous step>
+    </copy>
+    ````
+
+   If `Autoscaler` is listed as one of the addons you can go to the next lab in this workshop.  If not, proceed with the remaining tasks here to install it.
+
+## Task 2: Install the Autoscaler Addon
+
+Assuming the task before this indicated the addon was not installed you can now install the addon.  You can also optionally do the following steps manually in the web console for OKE.
+
+1. SSH to your Operater using the output from `terraform output`, example below.
+
+    ```bash
+    <copy>
+    ssh -J opc@<bastion public IP> opc@<operator private ip>
+    </copy>
+    ```
+
+2. Get the OCID of the `node_pool_workers` node pool. This is the pool that will run the Agones fleet in subsequent labs of this workshop.
+
+    ````shell
+    <copy>
+    kubectl get node -l oke.oraclecloud.com/pool.name=node_pool_workers -o json |grep node-pool-id
+    </copy>
+    ````
+
+   **This OCID is the Node Pool OCID that you will use in the next step.**
+
+3. The file [addon.json](./files/addon.json) will be used as an example. It's format is `<min nodes>:<max nodes>:<node pool id>`. It's important to remember that as your node pools change (renaming, changing terraform etc) their respective OCID's will change and you will need to update this config.
+
+   Create the config as `addon.json` and paste the contents from [addon.json](./files/addon.json). Replace from the file `<NODE POOL OCID>` with the OCID from the previous step above.
+
+    ```bash
+    <copy>
+    # using vim or nano
+    vim addon.json
+
+    # paste from addon.json into this new file and save
+    # Paste in the correct Node Pool OCID
+    </copy>
+    ```
+
+4. Install the addon using the newly created config file.  This should run without error and a resulting work request ID will be displayed.
+
+
+    ````shell
+    <copy>
+    oci ce cluster install-addon --addon-name ClusterAutoscaler --from-json file://addon.json --cluster-id <ocid of cluster>
+    </copy>
+    ````
+
+5. Verify there are no errors with the newly installed addon. The result should say `ACTIVE`.
+
+    ````shell
+    <copy>
+    oci ce cluster get-addon --addon-name ClusterAutoscaler --cluster-id <ocid of cluster> | grep lifecycle-state
+    </copy>
+    ````
+
+You may now **proceed to the next lab**
+
+## **Summary**
+
+You have installed the OKE Cluster Autoscaler addon on and configured it to watch the node pool that will be running the Agones fleet in subsequent labs of this workshop.
+
+## Learn More - *Useful Links*
+
+- [Working with the OKE Cluster Autoscaler Add-on](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengusingclusterautoscaler_topic-Working_with_Cluster_Autoscaler_as_Cluster_Add-on.htm)
+- ["oci ce cluster addon" documentation](https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.50.3/oci_cli_docs/cmdref/ce/cluster.html)
+
+## **Acknowledgements**
+
+ - **Author** - Marcellus Miles, Master Cloud Architect
+ - **Last Updated By/Date** - Marcellus Miles, Dec 2024
diff --git a/oke-with-agones/3-install-oke-autoscaler-addon/files/addon.json b/oke-with-agones/3-install-oke-autoscaler-addon/files/addon.json
@@ -0,0 +1,9 @@
+{
+  "addonName": "ClusterAutoscaler",
+  "configurations": [
+    {
+      "key": "nodes",
+      "value": "2:10:<NODE POOL OCID>"
+    }
+  ]
+}