[TT-12494] Fix flaky TestCacheEtag and related cache tests (#6508)

### **User description**
This considers some necessary improvements:

- [x] revert test to run e2e-combined (e2e only has some CI specific
issues when running, not seen on local)
- [x] add cancellation group to CI tests, new commits on PR cancel old
CI Tests run
- [x] pass only the merged .cov file to sonarcloud (not all of them)
- [x] https://tyktech.atlassian.net/browse/TT-12494 (tests issue,
deleting caches, defer)

The tests are flaky on account they delete the redis cache. Deleting a
redis cache from a test interferes with other tests that also read/write
cache data from redis.


___

### **PR Type**
enhancement, tests


___

### **Description**
- Added concurrency settings to the CI workflow to ensure only one
runner per PR and commit, canceling old runs on new commits.
- Updated the test command to `test:e2e-combined` to address CI-specific
issues.
- Changed the sonar coverage report path to use a merged `.cov` file
(`gateway-all.cov`).
- Simplified the package listing command in the test task configuration.


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>ci-tests.yml</strong><dd><code>Enhance CI workflow with
concurrency and test improvements</code></dd></summary>
<hr>

.github/workflows/ci-tests.yml

<li>Added concurrency settings to CI tests to cancel old runs on new
<br>commits.<br> <li> Changed test command to
<code>test:e2e-combined</code>.<br> <li> Updated sonar coverage report
path to <code>gateway-all.cov</code>.<br>


</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6508/files#diff-03609cb60b0c6e92fb771eb8787d6722b8c31ca4c03eabc788e147acd8c6fb43">+11/-2</a>&nbsp;
&nbsp; </td>

</tr>                    

<tr>
  <td>
    <details>
<summary><strong>test.yml</strong><dd><code>Simplify package listing
command in test task</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

.taskfiles/test.yml

- Simplified the package listing command in the test task.



</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6508/files#diff-f1fbe7f7f14888019b8845634ed008e1c43f6e5a5c0b2707336fc7f8e15a36fb">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>                    
</table></td></tr></tr></tbody></table>

___

> 💡 **PR-Agent usage**:
>Comment `/help` on the PR to get a list of all available PR-Agent tools
and their descriptions

---------

Co-authored-by: Tit Petric <tit@tyk.io>

.github/workflows/ci-tests.yml

@@ -17,6 +17,14 @@ on:
       - master
       - release-**
 
+# Only have one runner per PR, and per merged commit.
+#
+# - As a PR gets new commits, any old run jobs get cancelled (PR number)
+# - As a commit gets merged, it doesn't cancel previous running PR's (github.sha)
+concurrency:
+  group: ${{ github.event.pull_request.number || github.sha }}-ci-tests
+  cancel-in-progress: true
+
 env:
   PYTHON_VERSION: "3.11"
   PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python
@@ -86,6 +94,7 @@ jobs:
           pip install google
           pip install 'protobuf==4.24.4'
 
+          task --version
           task lint
 
           git add --all
@@ -117,7 +126,7 @@ jobs:
       - name: Run Gateway Tests
         id: ci-tests
         run: |
-          task test:e2e args="-race -timeout=15m"
+          task test:e2e-combined args="-race -timeout=15m"
           task test:coverage
 
       # golangci-lint actions *require* issues-exit-code=0 to pass data along to sonarcloud
@@ -144,7 +153,7 @@ jobs:
             -Dsonar.coverage.exclusions=**/*_test.go,**/mock/*
             -Dsonar.test.inclusions=**/*_test.go
             -Dsonar.tests=.
-            -Dsonar.go.coverage.reportPaths=coverage/*.cov
+            -Dsonar.go.coverage.reportPaths=coverage/gateway-all.cov
             -Dsonar.go.golangci-lint.reportPaths=golanglint.xml
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.taskfiles/test.yml

@@ -44,7 +44,7 @@ tasks:
       package:
         sh: go mod edit -json | jq .Module.Path -r
       packages:
-        sh: go list ./... | tail -n +2 | sed -e 's|{{.package}}|.|g'
+        sh: go list ./... | sed -e 's|{{.package}}|.|g'
     cmds:
       - defer: { task: services:down }
       - rm -rf coverage && mkdir -p coverage

docker/services/Taskfile.yml

@@ -5,7 +5,7 @@ tasks:
   up:
     desc: "Bring up services"
     cmds:
-      - docker compose up -d --remove-orphans
+      - docker compose up -d --remove-orphans --wait
 
   down:
     desc: "Tear down services"