SCALRCORE-32580 Agent > Restrict access to instance metadata (#64)

Co-authored-by: soltyssss <y.soltys@scalr.com>
Scalr · Oct 29, 2024 · a315697 · a315697
1 parent bd4008c
commit a315697
Show file tree

Hide file tree

Showing 5 changed files with 67 additions and 0 deletions.
diff --git a/charts/agent-k8s/CHANGELOG.md b/charts/agent-k8s/CHANGELOG.md
@@ -9,6 +9,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [UNRELEASED]
 
+### Added
+
+- Added `restrictMetadataService` option. When set to true, applies pod network policy that blocks outbound access to instance metadata service.
+
 ## [v0.5.21]
 
 ### Updated

diff --git a/charts/agent-k8s/README.md b/charts/agent-k8s/README.md
@@ -107,6 +107,25 @@ The EFS storage will be mounted in all worker containers at the `agent.data_home
 for Runs will inherit the EFS configuration. The controller will continue to use an ephemeral directory
 as its data home.
 
+## Restrict Access to VM Metadata Service
+
+The chart includes an optional feature to restrict the pods from accessing the VM metadata service at 169.254.169.254, which is common for both AWS and GCP environments.
+
+To enable it, use the `restrictMetadataService` option:
+
+```console
+$ helm upgrade ... \
+    --set restrictMetadataService=true
+```
+
+With this option enabled, a Kubernetes NetworkPolicy is applied to the agent pods that denies egress traffic to 169.254.169.254/32, blocking access to the VM metadata service. All other outbound traffic is allowed.
+
+### Limitations
+
+Ensure that your cluster is using a CNI plugin that supports egress NetworkPolicies. Example: Calico, Cilium, or native GKE NetworkPolicy provider for supported versions.
+
+If your cluster doesn't currently support egress NetworkPolicies, you may need to recreate it with the appropriate settings.
+
 ## Maintainers
 
 | Name | Email | Url |
@@ -155,6 +174,7 @@ as its data home.
 | resources.limits.memory | string | `"1024Mi"` |  |
 | resources.requests.cpu | string | `"250m"` |  |
 | resources.requests.memory | string | `"256Mi"` |  |
+| restrictMetadataService | bool | `false` | Apply NetworkPolicy to an agent pod that denies access to VM metadata service address (169.254.169.254) |
 | securityContext | object | `{"runAsGroup":0,"runAsUser":0}` | The Agent Pods security context. |
 | serviceAccount.annotations | object | `{}` | Annotations to add to the service account |
 | serviceAccount.create | bool | `true` | Specifies whether a service account should be created |

diff --git a/charts/agent-k8s/README.md.gotmpl b/charts/agent-k8s/README.md.gotmpl
@@ -99,6 +99,25 @@ The EFS storage will be mounted in all worker containers at the `agent.data_home
 for Runs will inherit the EFS configuration. The controller will continue to use an ephemeral directory
 as its data home.
 
+## Restrict Access to VM Metadata Service
+
+The chart includes an optional feature to restrict the pods from accessing the VM metadata service at 169.254.169.254, that is common for both AWS and GCP environments.
+
+To enable it, use the `restrictMetadataService` option:
+
+```console
+$ helm upgrade ... \
+    --set restrictMetadataService=true
+```
+
+With this option enabled, a Kubernetes NetworkPolicy is applied to the agent pods that denies egress traffic to 169.254.169.254/32, blocking access to the VM metadata service. All other outbound traffic is allowed.
+
+### Limitations
+
+Ensure that your cluster is using a CNI plugin that supports egress NetworkPolicies. Example: Calico, Cilium, or native GKE NetworkPolicy provider for supported versions.
+
+If your cluster doesn't currently support egress NetworkPolicies, you may need to recreate it with the appropriate settings.
+
 {{ template "chart.maintainersSection" . }}
 
 {{ template "chart.requirementsSection" . }}

diff --git a/charts/agent-k8s/templates/networkpolicy.yaml b/charts/agent-k8s/templates/networkpolicy.yaml
@@ -0,0 +1,21 @@
+{{- if .Values.restrictMetadataService }}
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: agent-k8s-network-policy
+  namespace: {{ .Release.Namespace | quote }}
+spec:
+  podSelector:
+    matchLabels:
+      {{- include "agent-k8s.selectorLabels" . | nindent 6 }}
+  policyTypes:
+    - Egress
+  egress:
+    - to:
+      - ipBlock:
+          # Allow all egress traffic by default
+          cidr: 0.0.0.0/0
+          except:
+            # Deny access to IMDS
+            - 169.254.169.254/32
+{{- end }}
diff --git a/charts/agent-k8s/values.yaml b/charts/agent-k8s/values.yaml
@@ -110,6 +110,9 @@ securityContext:
 # -- The Agent Pods annotations.
 podAnnotations: {}
 
+# -- Apply NetworkPolicy to an agent pod that denies access to VM metadata service address (169.254.169.254)
+restrictMetadataService: false
+
 # -- Provides the amount of grace time prior to the agent-k8s container being forcibly terminated when marked for deletion or restarted.
 terminationGracePeriodSeconds: 3660