plugins: Introduce phased plugin

[oshoval]

Phased plugin allows running jobs that will be considered as "phase 2"
only once both lgtm / approve are presented.

We are running 3 k8s versions for each sig on kubevirt, for each PR.
With this plugin we can declare the non latest one as "phase 2"
and run it only when the PR is entering the merge queue,
This will save CI resources.

A job will be declared as "phase 2", if it has always_run=false, optional=false,
without additional conditions such as run_if_changed, skip_if_only_changed.

Since the jobs are always_run=false it means that a potential race condition might happen
between running the job and merging.
For this we use require_manually_triggered_jobs which will block the PR entering
the merge queue / being merge by tide before all "phase 2" jobs finish.
It will also cause tide to auto run phase 2 when rerunning batches.

Notes:

• Doesn't support branches, since for branches we consider everything as stable phase 1.
  A follow-up PR will update the job forking to update phase 2 to be considered phase 1 on new branches.
• Works only for kubevirt/kubevirt.
• pull-kubevirt-e2e-k8s-1.27-ipv6-sig-network was chosen as phase 2 candid.
• Assuming both lgtm / approve were not added when PR was draft (else the plugin won't kick
  in once undrafted).

[oshoval]

/cc @brianmcarey @EdDev @dhiller

[brianmcarey]

Thanks @oshoval for this - it could be very useful. I just have a couple of questions inline.

[oshoval]

Rebased only

[oshoval]

Addressed comments, post submit is as bot-review now
and built latest image (updated manifest)
Thanks

[oshoval]

Removed branch support

on a follow PR we should update new branch forking to change phase 2 to be phase 1 for branches

[oshoval]

pull-project-infra-prow-deploy-test fails
i dont think its my PR ? it passed before small changes / rebase
or is it ?

[oshoval]

/retest

[oshoval]

worked now, there was a flake twice today

[oshoval]

first push - cosmetics: remove unused code
2nd push - rebase

[kubevirt-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dhiller

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [dhiller]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dhiller]

One thing I missed: you need to enable this plugin in the external_plugins config like here:

project-infra/github/ci/prow-deploy/kustom/base/configs/current/plugins/plugins.yaml

Line 522 in 0653b9d

- name: rehearse

[oshoval]

One thing I missed: you need to enable this plugin in the external_plugins config like here:

project-infra/github/ci/prow-deploy/kustom/base/configs/current/plugins/plugins.yaml

Line 522 in 0653b9d

- name: rehearse

It is here isn't it ?
https://github.com/kubevirt/project-infra/pull/3103/files#diff-3a088ffbe61dd30f930cd1855ab72a652fcea4a33c4b047eeb6df38f7a79ad51R531

[oshoval]

Pushed (separated):

• addressed the marked comments (commented on the rest) , added few tests cases, and refactor as table with some BeforeEach so it can be updated easier if needed
• rebased
• updated image

Thanks

/hold cancel
(we can do the rest in a follow up as mentioned)

[dhiller]

Sorry, I overlooked the external plugin misconfiguration in first pass. That needs to get fixed.

Other than than just a spelling mistake.

[oshoval]

Thanks

fixed comments
rebased
updated image

[oshoval]

created kubevirt/kubevirt for the plugin

[kubevirt-bot]

@oshoval: Updated the following 3 configmaps:

• job-config configmap in namespace kubevirt-prow at cluster default using the following files:
  • key kubevirt-presubmits.yaml using file github/ci/prow-deploy/files/jobs/kubevirt/kubevirt/kubevirt-presubmits.yaml
  • key project-infra-postsubmits.yaml using file github/ci/prow-deploy/files/jobs/kubevirt/project-infra/project-infra-postsubmits.yaml
• config configmap in namespace kubevirt-prow at cluster default using the following files:
  • key config.yaml using file github/ci/prow-deploy/kustom/base/configs/current/config/config.yaml
• plugins configmap in namespace kubevirt-prow at cluster default using the following files:
  • key plugins.yaml using file github/ci/prow-deploy/kustom/base/configs/current/plugins/plugins.yaml

In response to this:

Phased plugin allows running jobs that will be considered as "phase 2"
only once both lgtm / approve are presented.

We are running 3 k8s versions for each sig on kubevirt, for each PR.
With this plugin we can declare the non latest one as "phase 2"
and run it only when the PR is entering the merge queue,
This will save CI resources.

A job will be declared as "phase 2", if it has always_run=false, optional=false,
without additional conditions such as run_if_changed, skip_if_only_changed.

Since the jobs are always_run=false it means that a potential race condition might happen
between running the job and merging.
For this we use require_manually_triggered_jobs which will block the PR entering
the merge queue / being merge by tide before all "phase 2" jobs finish.
It will also cause tide to auto run phase 2 when rerunning batches.

Notes:

• Doesn't support branches, since for branches we consider everything as stable phase 1.
  A follow-up PR will update the job forking to update phase 2 to be considered phase 1 on new branches.
• Works only for kubevirt/kubevirt.
• pull-kubevirt-e2e-k8s-1.27-ipv6-sig-network was chosen as phase 2 candid.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.