OPI needs cluster-wide access to resources

[viovanov]

Description

The OPI service shouldn't require any cluster-wide permissions.

Steps to reproduce

Using a serviceaccount with access to the eirini namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: eirini-role
  namespace: {{ .Values.eirini.opi.namespace }}
rules:
...
- apiGroups:
  - apps
  resources:
  - statefulsets
  verbs:
  - create
  - update
  - delete
  - list
...

What was expected to happen

OPI should work.

What actually happened

Got an error.

Suggested fix (optional)

Only work with StatefulSets in the eirini namespace.

Additional information (optional)

{"timestamp":"2020-09-04T22:26:15.145143385Z","level":"error","so 
 urce":"handler","message":"handler.list-apps.bifrost-failed","dat 
 a":{"error":"failed to list desired LRPs: failed to list stateful 
 sets: statefulsets.apps is forbidden: User \"system:serviceaccoun 
 t:kubecf:opi\" cannot list resource \"statefulsets\" in API group 
  \"apps\" at the cluster scope","session":"57"}} 

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/174679495

The labels on this github issue will be updated when the story is started.

[jimmykarily]

Currently Eirini will try to deploy the application on whatever namespace is defined in the request (https://www.pivotaltracker.com/story/show/172890997). If we want to enable single-namespace operation, then validation should happen in various places (e.g. if the request asks for an app in a non-monitored namespace). I wonder what the use case behind letting Eirini deploy in multiple namespaces is and why that is not applicable to kubecf. Iirc is had something to do with implementing org/space separation using kube namespaces but I may be wrong. Someone else knows?

For reference, the cluster-wide permissions are needed because namespace is empty here: https://github.com/cloudfoundry-incubator/eirini/blob/master/k8s/client/clients.go#L87 (called here).

[jimmykarily]

This is the original issue that introduced the multi-namespace monitoring in Eirini: #90