fix: add separate workflow for CEL policies to unblock e2e tests

[Chandan-DK]

Related Issue(s)

Description

This PR resolves Strategy expansion exceeded 256 results for job 'chainsaw' error that is currently blocking the E2E tests from running.

A separate workflow file has been created for CEL policies and this PR makes use of composite actions to reuse steps in both the workflow files (test.yml and cel-test.yml)

Checklist

• [] I have read the policy contribution guidelines.
• [] I have added test manifests and resources covering both positive and negative tests that prove this policy works as intended.
• [] I have added the artifacthub-pkg.yml file and have verified it is complete and correct.

[Chandan-DK]

Made this PR by mistake. Was trying to find a way to avoid Strategy expansion exceeded 256 results for job 'chainsaw' error that is currently blocking the E2E tests from running. If separating the regular kyverno policies from the CEL policies with two workflow files seems like a suitable approach, I can reopen this and get it in shape. Closing for now.

[chipzoller]

Yes, I think this would be a good approach! Please do reopen!

[realshuting]

Hi @Chandan-DK - let us know if you are still interested in contribution!

[Chandan-DK]

Hi! I will get this PR in shape for review 🙌

[realshuting]

Thanks @Chandan-DK !

All new tests failed due to this:

   === ERROR

    admission webhook "validate-policy.kyverno.svc" denied the request: no unique match for kind GitRepository
l.go:53: ���������| 18:25:20 | verify-git-repositories | 01 - Create policy and verify  | CREATE    | ERROR | kyverno.io/v1/ClusterPolicy @ verify-git-repositories
    === ERROR
    admission webhook "validate-policy.kyverno.svc" denied the request: no unique match for kind GitRepository

Looks like something is missing, can you take a look?

[Chandan-DK]

Thanks @Chandan-DK !

All new tests failed due to this:

   === ERROR

    admission webhook "validate-policy.kyverno.svc" denied the request: no unique match for kind GitRepository
l.go:53: ���������| 18:25:20 | verify-git-repositories | 01 - Create policy and verify  | CREATE    | ERROR | kyverno.io/v1/ClusterPolicy @ verify-git-repositories
    === ERROR
    admission webhook "validate-policy.kyverno.svc" denied the request: no unique match for kind GitRepository

Looks like something is missing, can you take a look?

verify-git-repositories cel policy is not being created due to this issue (kyverno/kyverno#10313). I have removed this policy for now.

[chipzoller]

Could we rename the .chainsaw-test directory of the problematic policy rather than just deleting it?

[Chandan-DK]

Sure, but there will still be noise in the CI since Chainsaw executes the chainsaw-test.yaml file. We could rename this file too, to prevent noise. What do you think?

[chipzoller]

I would ask @eddycharly what's the most reliable thing to do here. I'm not sure what the current recommended approach is for Chainsaw given its present state.

[Chandan-DK]

Chainsaw by default looks for the chainsaw-test.yaml file inside every folder. We could rename the .chainsaw-test directory of the problematic policy to something like .chainsaw-test-rename-after-issue-10313-fix to make it easy to identify this problematic policy in the future. Changing the directory name won't stop the chainsaw-test.yaml file that is present in it from being executed. So, we should probably name the file this way chainsaw-test-rename-after-issue-10313-fix.yaml and avoid a failing chainsaw test for the time being.

By the way, the policies that have been deleted due to this issue are being tracked here. I have been referencing the link of the issue in my commits when deleting policies. We could use those commits to help add the policies back when the issue is resolved.

[chipzoller]

Ok, that sounds good. Let's give it a try.

[chipzoller]

Much thanks! I'm going to go ahead and merge this so we can run our e2e tests again.

[realshuting]

Good job @Chandan-DK !

[Chandan-DK]

Thanks @realshuting @chipzoller !