Add chainsaw support for e2e testing

[eddycharly]

Chainsaw was created because kuttl stopped to be actively maintained (it's actually a single maintainer now).

We are also solving most of the kuttl limitations that exist today and wrote a command to automatically migrate from kuttl to chainsaw. It supercharges e2e tests with better logs, config maps assertions, assertions trees and many more things 💪

This is a relatively young project but it has already been adopted by a couple of e2e tests heavy projects (kyverno, keptn, fairwinds rbac-manager, grafana-operator, redis-operator and others).

To give more context, you can see similar PRs at:

• chore: replace kuttl tests with chainsaw keptn/lifecycle-toolkit#3000
• chore: migrate from kuttl to chainsaw open-telemetry/opentelemetry-operator#2630
• feat: add chainsaw scorecard test support operator-framework/operator-sdk#6676
• chore: migrate from kuttl to chainsaw grafana/tempo-operator#786 in this PR the tempo-operator maintainers asked some good questions and I explained the current status, that I suggest taking a look at.

Sadly, kuttl haven't done a release since jan 2023.

Here you can find a good video showcasing chainsaw: https://www.youtube.com/watch?v=Ejof-wtAdQM&ab_channel=CamilaMacedo

If you need more infos 👇

• GitHub: https://github.com/kyverno/chainsaw
• Docs: https://kyverno.github.io/chainsaw
• Slack: https://kubernetes.slack.com/archives/C067LUFL43U

[jeanduplessis]

Thanks for bringing this to our attention @eddycharly - we have plans to invest in internal improvements in Uptest soon and will evaluate chainsaw as part of this effort.

[eddycharly]

Hi @jeanduplessis let me know if you want to discuss chainsaw, we can organise a meeting if needed.

Chainsaw was adopted by a bunch of leading orgs like otel, kyverno, keptn and grafana and we are working with argo, percona, operator-sdk, kubebuilder and other folks.

I'm convinced it will become the commonly adopted tool in the coming months, the project is very active and has a nice roadmap ahead :)

[vfarcic]

@jeanduplessis please invite me to the meeting. I've been exploring Chainsaw for testing my compositions and will likely switch to it from KUTTL.

[eddycharly]

@vfarcic glad to hear that :)

v0.1.4 introduced resource templating, v0.1.6 introduced multi cluster support, and we have another great feature coming to pass data from one operation to the next ones (think about operation outputs) :)

That's probably going to close the v0.1.x series, v0.2.x will be all about metrics and making chainsaw able to natively assert metrics are working correctly.
This is really something i want to tackle, metrics are not well tested enough IMHO.

[jeanduplessis]

With this PR we are proposing moving to chainsaw as the underlying test framework for Uptest. I will close the issue here for now.

[eddycharly]

Thank you @jeanduplessis, I didn't know about this PR. Awesome news !

[jeanduplessis]

Thank you @eddycharly, for advocating for it and bringing it to our attention.
(We recently moved Uptest upstream to the crossplane org which is probably why you didn't see the PR)