Konnect entities: All entities get deleted when referenced ControlPlane gets deleted

[randmonkey]

Current Behavior

When a KonnectGatewayControlPlane is deleted, the control plane is deleted on the Konnect side and all attached entities (services, routes, consumers, ...) gets deleted as well. Then, all corresponding k8s resources (KongService,KongRoute,KongConsumer,...) gets deleted silently.

Expected Behavior

Similar to #786, If cascade deletion is expected, we should add logs to tell that the resource are deleted.

Steps To Reproduce

Operator Version

kubectl version

[pmalek]

We have to consider if we'd like to support a use case where users having CP and entities configured within it:

• delete a CP
• create a CP (with the same name and type)

Currently, after the CP deletion, users will get all their entities from within that CP deleted. Recreating the CP will not have the same config as all its configuration entities are gone at this point.

Changing the behavior to not delete these entities would:

• make the configuration entities have their status updated
• allow recreating the CP which would get all the configuration that previous CP had
• make Konnect entities work more seamlessly with git-ops tools: there would be no drift between what's defined in source control and what's in cluster

cc: @mheap

[mheap]

This is a tough one. Cascade delete is what a lot of people would expect, but I'm not sure that's true in a cluster.

I think that I'd expect non-CP resources to remain in the cluster with an updated status, and if they recreate the CP they will start to reconcile again.

[pmalek]

As agreed on slack https://kongstrong.slack.com/archives/C079GK3KB9R/p1730102565181899 we're ok with shipping 1.4 without this change.

Adding to 1.5.x milestone to fix after 1.4 gets released.

[lahabana]

@ajessup is gathering more data to figure what we should do here.

Let's ship with "status" update instead of cascade deletion unless we hear otherwise from @ajessup