Skip to content

Commit

Permalink
Update some scripts and docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Jaykul committed Oct 5, 2023
1 parent 44609f0 commit 231f324
Show file tree
Hide file tree
Showing 6 changed files with 78 additions and 8 deletions.
13 changes: 13 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"cSpell.words": [
"bitnami",
"crds",
"fluxcd",
"Kustomization",
"kustomize",
"podinfo",
"poshcode",
"traefik",
"valuesfrom"
]
}
19 changes: 11 additions & 8 deletions Initialize-Azure.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,27 @@
#>
[CmdletBinding()]
param(
# The base name to use. E.g. the "cluster" name
[Parameter(Mandatory = $true)]
[string]$baseName,

# If set, will remove the existing app and service principal, so we can recreate it.
[switch]$RemoveExisting,

# The resource group to create. E.g. "rg-poshcode"
[string]$resourceGroupName = "rg-poshcode",

# The location to create the resource group in. E.g. "eastus"
[string]$location = "eastus",

# The service name to use. E.g. "rg-poshcode-deploy"
[string]$serviceName = "rg-poshcode-deploy",

# The repo to set secrets for. E.g. "PoshCode/cluster"
[string]$repo = "PoshCode/cluster"
)

# The resource group to create. E.g. "rg-cluster"
$resourceGroupName = "rg-${baseName}"

# The service name to use. E.g. "rg-cluster-deploy"
$serviceName = "rg-${baseName}-deploy"

# Register a bunch of preview features
# "CiliumDataplanePreview" is not working as far as I can tell
Get-AzProviderFeature -ProviderNamespace Microsoft.ContainerService -OutVariable enabledFeatures
foreach ($feature in "AKS-KedaPreview", "AKSNetworkModePreview", "AzureOverlayPreview",
"EnableBlobCSIDriver", "EnableNetworkPolicy", "EnableWorkloadIdentityPreview", "NRGLockdownPreview",
Expand Down Expand Up @@ -68,7 +71,7 @@ gh secret set --repo https://github.com/$repo AZURE_SUBSCRIPTION_ID -b $ctx.Subs
gh secret set --repo https://github.com/$repo AZURE_RG -b $resourceGroupName
# gh secret set --repo https://github.com/$repo USER_OBJECT_ID -b $spId

# This stuff doesn't HAVE to be done...
# Create an AD Group to be administrators of the cluster:
$admins = (Get-AzADGroup -Filter "DisplayName eq 'AksAdmins'") ??
(New-AzADGroup -DisplayName AksAdmins -MailNickname AksAdmins -Description "Kubernetes Admins")

Expand Down
44 changes: 44 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# PoshCode k8s Cluster

This repo has a full bicep deployment for a Kubernetes Cluster, including a github workflow to deploy it, and full yaml to configure all the applications on it.

There are two parts:

1. The **infrastructure** deployment, written in Azure Bicep
2. The **GitOps configuration** (in yaml, in the `clusters`, `system`, and `apps` folders)

## Infrastructure Deployment

I've written my own template for deploying AKS, and it's in the `Infrastructure` folder. It's written in [Azure Bicep](https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview), and is relatively opinionated, since we're using Flux for GitOps.

### Prerequisites

1. Enable some pre-release features in your Azure tenant
2. Create a resource group in Azure
3. Create a service account in Azure for automation
4. Create secrets in github for authentication as that service account

See [Initialize-Azure](./Initialize-Azure.ps1)` for details. You might call it like this:

```PowerShell
./Initialize-Azure -baseName $name
```

### Deploying

Basically, you're going to run something like this, except we have a [workflow for that](.github/workflows/deploy.yaml).


```PowerShell
$Deployment = @{
Name = "aks-$(Get-Date -f yyyyMMddThhmmss)"
ResourceGroupName = "rg-$name"
TemplateFile = ".\infrastructure\Cluster.bicep"
TemplateParameterObject = @{
baseName = "$name"
adminId = (Get-AzADGroup -Filter "DisplayName eq 'AksAdmins'").Id
}
}
New-AzResourceGroupDeployment @Deployment
```
1 change: 1 addition & 0 deletions clusters/poshcode/apps.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ spec:
path: ./apps
prune: true
wait: true

commonMetadata:
annotations:
poshcode.org/tenant: sre-team
2 changes: 2 additions & 0 deletions clusters/poshcode/services.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ spec:
kind: GitRepository
name: bootstrap
path: ./system/services
decryption:
provider: sops
prune: true
wait: true
commonMetadata:
Expand Down
7 changes: 7 additions & 0 deletions infrastructure/Cluster.bicep
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,8 @@ module flux 'modules/flux.bicep' = {
baseName: baseName
gitOpsRepositoryUrl: gitOpsRepositoryUrl
}
// technically not, but we're using the deploy to cause some delays...
dependsOn: [ aks, fluxId ]
}

// module alerts 'modules/metricAlerts.bicep' = {
Expand Down Expand Up @@ -297,6 +299,8 @@ module iam_flux_crypto 'modules/resourceRoleAssignment.bicep' = {
resourceId: keyVault.outputs.id
roleName: 'Key Vault Crypto User'
}
// technically not, but we're using the deploy to cause a delay...
dependsOn: [ flux ]
}

@description('Flux release namespace')
Expand All @@ -314,6 +318,9 @@ output userAssignedIdentityPrincipalId string = kubeletId.outputs.principalId
@description('User Assigned Identity Client ID, used for application config (so we can use this identity from code)')
output userAssignedIdentityClientId string = kubeletId.outputs.clientId

@description('Uri for the sops-key to be used for secret encryption')
output sopsKeyId string = keyVault.outputs.sopsKeyId

// output LogAnalyticsName string = logAnalytics.name
// output LogAnalyticsGuid string = logAnalytics.properties.customerId
// output LogAnalyticsId string = logAnalytics.id

0 comments on commit 231f324

Please sign in to comment.