Swiftv2 pipeline branch #4099
Swiftv2 pipeline branch #4099
Conversation
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: sivakami-projects <126191544+sivakami-projects@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: sivakami-projects <126191544+sivakami-projects@users.noreply.github.com>
…ween vnet 1 subnet 1 and vnet 1 subnet2.
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a new Azure DevOps pipeline infrastructure for long-running tests on AKS Swift v2 clusters. The pipeline automates the provisioning of dual AKS clusters with custom networking configurations including VNet peering, private endpoints, and network security groups.
Key changes:
- Parameterized pipeline template with sequential job dependencies for infrastructure provisioning
- Six bash scripts for creating resource groups, AKS clusters, VNets, peerings, storage accounts, NSGs, and private endpoints
- Parallel AKS cluster creation to optimize deployment time
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
.pipelines/swiftv2-long-running/template/long-running-pipeline-template.yaml |
Defines the pipeline template with three jobs: resource group creation, parallel AKS cluster provisioning, and networking/storage setup |
.pipelines/swiftv2-long-running/scripts/create_vnets.sh |
Creates four VNets with multiple subnets using predefined CIDR ranges |
.pipelines/swiftv2-long-running/scripts/create_storage.sh |
Provisions two storage accounts with security configurations and exports their names as pipeline variables |
.pipelines/swiftv2-long-running/scripts/create_peerings.sh |
Establishes bidirectional VNet peerings between three of the four VNets |
.pipelines/swiftv2-long-running/scripts/create_pe.sh |
Creates private DNS zone, links it to VNets, and provisions private endpoint for storage account |
.pipelines/swiftv2-long-running/scripts/create_nsg.sh |
Creates NSG with bidirectional deny rules between two subnets in the same VNet |
.pipelines/swiftv2-long-running/scripts/create_aks.sh |
Creates two AKS clusters in parallel, each with a default node pool and a high-NIC node pool |
.pipelines/swiftv2-long-running/pipeline.yaml |
Main pipeline configuration with parameters for subscription, location, resource group, and VM SKUs |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
/azp run Azure Container Networking PR |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| az network vnet subnet create -g "$RG" --vnet-name "$VNET_A1" -n pe --address-prefix "$A1_PE" --output none \ | ||
| && echo "Created $VNET_A1 with subnet pe" | ||
|
|
||
| # A2 |
There was a problem hiding this comment.
Also I just realized, what is the purpose of vnet 2,3 and b1?
I see A1 has a storage account attached to it via a private endpoint to A1's subnet, but what are A2,3 and B1 for? (I see we peer them all together, but are there supposed to be any resources on A2,3,B1?
There was a problem hiding this comment.
Yeah I guess a follow-up question, what is the purpose of VnetA1 as well?
Maybe I'm missing the over-arching theme here, I see we make 2 AKS clusters, and then separately, make these VNets, 1 storage, 1 NSG, 1 PE, and peer the vnets together
Is this towards something larger? What is the bigger picture?
There was a problem hiding this comment.
Bigger picture is we are copying singularity's setup. Vnet A1, A2, A3 belong to customer and hence they are all peered. As we create workloads, we will add pods to all these vnets. Vnet A1, A2 and A3 are peered. Vnet A1 has the subnet pe with NIC to access private endpoint to the storage account, Vnet A2 and Vnet A3 also access the storage account through this same subnet as they are peered. While creating the private endpoint, i have created private endpoint link for all three Vnets A1, A2 and A3.
Vnet B1 has a storage account SA2. We are not going to use a private endpoint for this.
|
/azp run Azure Container Networking PR |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| local CLUSTER=$1 | ||
| echo "==> Creating AKS cluster: $CLUSTER" | ||
|
|
||
| az aks create -g "$RG" -n "$CLUSTER" -l "$LOCATION" \ |
There was a problem hiding this comment.
This, and the command below it, might be best to move to https://github.com/Azure/azure-container-networking/blob/master/hack/aks/Makefile as a Make target
See
See https://github.com/Azure/azure-container-networking/blob/6c232c8618594bfda4f8b9fbe23b6a6bf422c7a9/.pipelines/pipeline.yaml#L321C8-L321C53 for clusterType variable such as dualstack-byocni-nokubeproxy-up
There was a problem hiding this comment.
Yeah, as it is, we would not be including the ip-tags we need to avoid SFI items
So I think we should use the cniv1-up target in that Makefile I linked, and add a new target linux-nodepool-up
| && echo "Storage account $SA created successfully." | ||
| done | ||
|
|
||
| echo "All storage accounts created successfully." |
There was a problem hiding this comment.
generic note for all scripts/resources: check if we need to add some step to validate the resource is created, and if not successful we get the error message printed in logs
There was a problem hiding this comment.
we might need this when you add the tests for reaching to storage account endpoint
| # ------------------------------------------------------------ | ||
| - job: NetworkingAndStorage | ||
| displayName: "Networking and Storage Setup" | ||
| dependsOn: CreateResourceGroup |
There was a problem hiding this comment.
will these stages be parallel? job 2 & 3. both depend on CreateResourceGroup
3 participants
test:
This PR introduces a new Azure DevOps pipeline for running long-term tests on AKS Swift v2 clusters.
The pipeline automates the creation of AKS clusters and the necessary networking infrastructure.
Pipeline link
Create 2 AKS clusters and attaches node pool with 2 nodes with 8 NICs on each node. All are AKS managed Linux nodes.
Creates 4 VNets and subnets.
Peers three of the four VNets.
Creates 2 storage accounts with private endpoints.
Adds NSG rules to prevent network connectivity between pods on two subnets in the same Vnet.
Initial PR - #4092
Creates a parameterized pipeline template that orchestrates AKS cluster creation and networking setup
Implements bash scripts to provision two AKS clusters, virtual networks, network peerings, storage accounts, and network security groups
Configures parallel execution for AKS cluster creation to optimize deployment time