Update README to reflect changes to terraform version and removal of cleanup steps

Author: marwan37

tests/integration/README.md

@@ -1,6 +1,7 @@
 # Integrating Local Testing of AWS Deployments in MLStacks Using LocalStack
 
 ## Prerequisites
+
 - Docker
 - AWS CLI
 - Terraform
@@ -14,11 +15,13 @@
 Installation can be done via Homebrew, PIP, Docker CLI, or Docker Compose. For detailed instructions, refer to the [LocalStack Installation Guide](https://docs.localstack.cloud/getting-started/installation/).
 
 **Homebrew**
+
 ```bash
 brew install localstack/tap/localstack-cli
 ```
 
 **PIP**
+
 ```bash
 python -m venv .venv
 source .venv/bin/activate
@@ -30,6 +33,7 @@ pip install localstack
 First, ensure Docker is running. Then, you can start LocalStack using the LocalStack CLI, Docker or Docker Compose.
 
 **LocalStack CLI**
+
 ```bash
 localstack start -d
 ```
@@ -50,11 +54,9 @@ docker-compose -f docker-compose.localstack.yml up -d
 
 > You may customize this file as needed. If you don't specify any services, LocalStack defaults to running all supported AWS services. Refer to the [LocalStack Docker Compose guide](https://docs.localstack.cloud/getting-started/installation/#starting-localstack-with-docker-compose) for more details.
 
-
-
 ### Interacting with LocalStack
 
-Once LocalStack is running, interact with it using `aws` commands by specifying the `endpoint-url`. 
+Once LocalStack is running, interact with it using `aws` commands by specifying the `endpoint-url`.
 For instance, to create an S3 bucket:
 
 ```bash
@@ -64,11 +66,13 @@ aws --endpoint-url=http://localhost:4566 s3 mb mybucket
 To avoid having to specify the `endpoint-url` with each command, you can install the [awscli-local](https://github.com/localstack/awscli-local) package. It provides the `awslocal` command, which automatically targets the LocalStack endpoint.
 
 So you can simply do this...
+
 ```bash
 awslocal s3 mb s3://test-bucket
 ```
 
 Instead of this...
+
 ```bash
 aws --endpoint-url=http://localhost:4566 s3 mb mybucket
 ```
@@ -82,11 +86,13 @@ pip install awscli-local
 #### Usage examples
 
 List all buckets:
+
 ```bash
 awslocal s3 ls
 ```
 
 Create a DynamoDB table:
+
 ```bash
 awslocal dynamodb create-table \
     --table-name test-table \
@@ -97,17 +103,16 @@ awslocal dynamodb create-table \
 ```
 
 List DynamoDB tables:
+
 ```bash
 awslocal dynamodb list-tables --region eu-north-1
 ```
 
-> **Note:**
-> **Most AWS services require specifying the region when using LocalStack, except for S3, which is globally accessible.**
-
+> **Note:** > **Most AWS services require specifying the region when using LocalStack, except for S3, which is globally accessible.**
 
 ## Provisioning AWS resources with Terraform + LocalStack
 
-To simulate the deployment of AWS resources locally, we utilize Terraform in conjunction with LocalStack. 
+To simulate the deployment of AWS resources locally, we utilize Terraform in conjunction with LocalStack.
 
 ### Terraform Configuration
 
@@ -136,21 +141,20 @@ force_destroy     = true
 
 This documentation describes the GitHub Actions workflow for LocalStack-based AWS integration testing within MLStacks. It features two primary jobs: `aws_remote_state_integration_test` and `aws_modular_integration_test`, aimed at provisioning and validating AWS resources in a local setup.
 
-
 ### Implementing Terraform Overrides
 
 For simulating AWS resources in our integration tests, we have two approaches:
 
 1. Manual configuration using override files.
 2. Utilizing `tflocal` for a more automated setup.
 
-#### [Manual Configuration](https://docs.localstack.cloud/user-guide/integrations/terraform/#manual-configuration) 
+#### [Manual Configuration](https://docs.localstack.cloud/user-guide/integrations/terraform/#manual-configuration)
 
-We have adopted the manual configuration method using an `_override.tf` file to specify provider settings tailored for LocalStack. 
-This approach allows us to directly manipulate how Terraform interacts with AWS services. 
+We have adopted the manual configuration method using an `_override.tf` file to specify provider settings tailored for LocalStack.
+This approach allows us to directly manipulate how Terraform interacts with AWS services.
 
-> Terraform inherently recognizes any file ending in `override.tf` as an override file, allowing you to alter configurations without modifying the primary Terraform files. 
-Detailed guidance on employing override files is available in the [Terraform Override Files Documentation](https://developer.hashicorp.com/terraform/language/files/override).
+> Terraform inherently recognizes any file ending in `override.tf` as an override file, allowing you to alter configurations without modifying the primary Terraform files.
+> Detailed guidance on employing override files is available in the [Terraform Override Files Documentation](https://developer.hashicorp.com/terraform/language/files/override).
 
 #### Alternative: [Using `terraform-local`](https://docs.localstack.cloud/user-guide/integrations/terraform/)
 
@@ -170,6 +174,7 @@ on:
   workflow_call:
   workflow_dispatch:
 ```
+
 ## Jobs Overview
 
 ### AWS Modular Integration Test
@@ -200,12 +205,12 @@ services:
 ```yml
 steps:
   - name: Checkout Repository
-    uses: actions/checkout@v2
+    uses: actions/checkout@v4.1.1
 
   - name: Setup Terraform
-    uses: hashicorp/setup-terraform@v2
+    uses: hashicorp/setup-terraform@v3.0.0
     with:
-      terraform_version: 1.3.4
+      terraform_version: 1.6.0
 
   - name: Copy Override File
     run: |
@@ -238,33 +243,14 @@ steps:
       ABSOLUTE_PATH="${GITHUB_WORKSPACE}/src/mlstacks/terraform/aws-modular/${STACK_YAML_PATH}"
       ../../../../tests/integration/aws-modular/verify_stack.sh "$ABSOLUTE_PATH"
     working-directory: src/mlstacks/terraform/aws-modular
-
-  - name: Cleanup Resources for aws-modular
-    run: |
-      cd src/mlstacks/terraform/aws-modular
-      terraform destroy -auto-approve -var-file="../../../../tests/integration/aws-modular/local.tfvars"
-      rm -rf .terraform
-      rm -rf volume
-      rm -f .terraform.lock.hcl
-      rm -f terraform.tfstate
-      rm -f terraform.tfstate.backup
-      rm -f localstack_providers_override.tf
-      rm -f aws_modular_stack_*.yaml
-
-  - name: Ensure _override.tf Deletion
-    if: always()
-    run: |
-      rm -f src/mlstacks/terraform/aws-modular/_override.tf
-    working-directory: ${{ github.workspace }}
 ```
+
 - **Copy Override File**: Copies the `_override.tf` file into the `aws-modular` directory to ensure Terraform operations target the LocalStack environment.
 - **Apply Terraform Configuration**: Navigates to the `aws-modular` directory, initializes Terraform with LocalStack as the backend, validates the configuration, and applies it using a `.tfvars` file.
 - **Refresh Terraform State**: Refreshes the state file to ensure the latest state is accurately reflected, including the generation of the YAML file.
 - **Output Stack YAML Path**: Utilizes `terraform-bin` instead of the standard `terraform` command to accurately capture and output the `stack-yaml-path`. This change was required to bypass `terraform_wrapper`, and ensures Terraform commands execute directly without any abstractions.
 - **Run Tests to Verify Resource Provisioning**:This step captures the YAML file's path, generated by the Terraform apply process, into a variable. It then utilizes this path to run a bash script to verify the provisioning and configuration of resources.
-    > **Note: Using an absolute path is essential here due to the test script's location in `tests/integration`.** 
-- **Cleanup Resources**: Handles the teardown of all resources and the removal of Terraform-related files to ensure no residual state or configurations are left behind.
-- **Ensure `_override.tf` Deletion**: Ensures the copied `_override.tf` file is removed after job completion.
+  > **Note: Using an absolute path is essential here due to the test script's location in `tests/integration`.**
 
 ### AWS Remote State Integration Test
 
@@ -289,18 +275,17 @@ services:
       AWS_SECRET_ACCESS_KEY: test
 ```
 
-
 #### Steps
 
 ```yml
 steps:
   - name: Checkout Repository
-    uses: actions/checkout@v2
+    uses: actions/checkout@v4.1.1
 
   - name: Setup Terraform
-    uses: hashicorp/setup-terraform@v2
+    uses: hashicorp/setup-terraform@v3.0.0
     with:
-      terraform_version: 1.3.4
+      terraform_version: 1.6.0
 
   - name: Copy Override File
     run: |
@@ -320,31 +305,12 @@ steps:
       AWS_ACCESS_KEY_ID: test
       AWS_SECRET_ACCESS_KEY: test
       AWS_DEFAULT_REGION: eu-north-1
-
-  - name: Cleanup Resources for aws-remote-state
-    run: |
-      cd src/mlstacks/terraform/aws-remote-state
-      terraform destroy -auto-approve -var-file="../../../../tests/integration/aws-remote-state/local.tfvars"
-      rm -rf .terraform
-      rm -rf volume
-      rm -f .terraform.lock.hcl
-      rm -f terraform.tfstate
-      rm -f terraform.tfstate.backup
-      rm -f localstack_providers_override.tf
-
-  - name: Ensure _override.tf Deletion
-    if: always()
-    run: |
-      rm -f src/mlstacks/terraform/aws-remote-state/_override.tf
-    working-directory: ${{ github.workspace }}
 ```
 
 - **Setup Terraform**: Prepares the Terraform environment.
 - **Copy Override File**: Copies the `_override.tf` file into the `aws-remote-state` directory to ensure Terraform operations target the LocalStack environment.
 - **Apply Terraform Configuration**: Navigates to the `aws-remote-state` directory, initializes Terraform with LocalStack as the backend, validates the configuration, and applies it using a `.tfvars` file.
 - **Run Tests**: Executes a bash script to verify resource provisioning.
-- **Cleanup Resources**: Destroys provisioned resources and cleans up Terraform-related files.
-- **Ensure `_override.tf` Deletion**: Ensures the copied `_override.tf` file is removed after job completion.
 
 ## Conclusion
 
@@ -366,4 +332,4 @@ In this documentation, we covered:
 - [terraform-local package](https://github.com/localstack/terraform-local)
 - [Override Files Terraform Documentation](https://developer.hashicorp.com/terraform/language/files/override)
 - [`.tfvars` documentation](https://developer.hashicorp.com/terraform/language/values/variables#variable-definitions-tfvars-files)
-- [awscli-local package](https://github.com/localstack/awscli-local)
+- [awscli-local package](https://github.com/localstack/awscli-local)