Skip to content

Commit

Permalink
Revert "[MANOPD-72386] Add new functional Kubernetes audit (#101)" (#123
Browse files Browse the repository at this point in the history
)

This reverts commit 160d50a.
  • Loading branch information
koryaga authored Feb 22, 2022
1 parent 197101b commit 984dbdc
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 134 deletions.
67 changes: 4 additions & 63 deletions documentation/Installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ This section provides information about the inventory, features, and steps for i
- [etc_hosts](#etc_hosts)
- [coredns](#coredns)
- [loadbalancer](#loadbalancer)
- [audit-Kubernetes Policy](#audit-Kubernetes Policy)
- [RBAC psp](#rbac-psp)
- [Configuring Admission Controller](#configuring-admission-controller)
- [Configuring OOB Policies](#configuring-oob-policies)
Expand Down Expand Up @@ -903,8 +902,7 @@ By default, the installer uses the following parameters:
|apiServer.certSANs|List with all nodes internal IPs, external IPs and names|
|apiServer.extraArgs.enable-admission-plugins|`NodeRestriction`|
|apiServer.extraArgs.profiling|`false`|
|apiServer.extraArgs.audit-log-path|`/var/log/kubernetes/audit/audit.log`|
|apiServer.extraArgs.audit-policy-file|`/etc/kubernetes/audit-policy.yaml`|
|apiServer.extraArgs.audit-log-path|`/var/log/apiserver/audit.log`|
|apiServer.extraArgs.audit-log-maxage|`30`|
|apiServer.extraArgs.audit-log-maxbackup|`10`|
|apiServer.extraArgs.audit-log-maxsize|`100`|
Expand Down Expand Up @@ -2056,65 +2054,9 @@ services:

**Warning**: If the changes to the hosts `sysctl` configurations are detected, a reboot is scheduled. After the reboot, the new parameters are validated to match the expected configuration.

#### audit-Kubernetes Policy
#### audit

*Installation task*: `prepare.system.audit.configure_policy`

*Can cause reboot*: No

*Can restart service*: Always yes

*OS specific*: No.

The `audit-Kubernetes Policy` section controls the audit setting for Kubernetes cluster. . By default, the following key-values are configured:
```yaml
services:
audit:
cluster_policy:
apiVersion: audit.k8s.io/v1
kind: Policy
omitStages:
- "RequestReceived"
rules:
- level: Request
resources:
- group: "" # core
- group: "admissionregistration.k8s.io"
- group: "apps"
- group: "authentication.k8s.io"
- group: "authorization.k8s.io"
- group: "autoscaling"
- group: "batch"
- group: "certificates.k8s.io"
- group: "extensions"
- group: "networking.k8s.io"
- group: "policy"
- group: "rbac.authorization.k8s.io"
- group: "settings.k8s.io"
- group: "storage.k8s.io"
```
**Note**: Each user can set up their own rules by specifying them in cluster.yaml

Example:

```yaml
services:
audit:
cluster_policy:
apiVersion: audit.k8s.io/v1
kind: Policy
omitStages:
- "RequestReceived"
rules:
- level: Metadata
resources:
- group: "authentication.k8s.io"
- group: "authorization.k8s.io"
```
#### audit-System Policy

*Installation task*: `prepare.system.audit.daemon`
*Installation task*: `prepare.system.audit`

*Can cause reboot*: No

Expand Down Expand Up @@ -4208,8 +4150,7 @@ The following is the installation tasks tree:
* **sysctl** - Configures Linux Kernel parameters. For more information about parameters for this task, see [sysctl](#sysctl).
* **audit**
* **install** - Installs auditd daemon on Ubuntu/Debian nodes.
* **configure_daemon** - Configures Linux audit rules. For more information about parameters for this task, see [audit](#audit).
* **configure_policy** - Configures Kubernetes audit rules. For more information about parameters for this task, see [audit-Kubernetes Policy](#audit-Kubernetes Policy)
* **configure** - Configures Linux audit rules. For more information about parameters for this task, see [audit](#audit).
* **cri**
* **install** - Installs the container runtime. For more information about parameters for this task, see [CRI](#cri).
* **configure** - Configures the container runtime. For more information about parameters for this task, see [CRI](#cri).
Expand Down
39 changes: 5 additions & 34 deletions kubemarine/procedures/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,18 +13,17 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import os.path

from collections import OrderedDict
import fabric
import yaml
import ruamel.yaml
import io

from kubemarine.core.errors import KME
from kubemarine import system, sysctl, haproxy, keepalived, kubernetes, plugins, \
kubernetes_accounts, selinux, thirdparties, psp, audit, coredns, cri, packages, apparmor
from kubemarine.core import flow, utils
from kubemarine.core.executor import RemoteExecutor
from kubemarine.core.yaml_merger import default_merger


def system_prepare_check_sudoer(cluster):
for host, node_context in cluster.context['nodes'].items():
Expand Down Expand Up @@ -111,37 +110,10 @@ def system_install_audit(cluster):
cluster.log.debug(group.call(audit.install))


def system_prepare_audit_daemon(cluster):
def system_prepare_audit(cluster):
group = cluster.nodes['master'].include_group(cluster.nodes.get('worker')).get_new_nodes_or_self()
cluster.log.debug(group.call(audit.apply_audit_rules))

def system_prepare_policy(cluster):
"""
Task generates rules for logging kubernetes
"""
audit_log_dir = os.path.dirname(cluster.inventory['services']['kubeadm']['apiServer']['extraArgs']['audit-log-path'])
audit_policy_dir = os.path.dirname(cluster.inventory['services']['kubeadm']['apiServer']['extraArgs']['audit-policy-file'])
audit_file_name = cluster.inventory['services']['kubeadm']['apiServer']['extraArgs']['audit-policy-file']
cluster.nodes['master'].run(f"sudo mkdir -p {audit_log_dir} && sudo mkdir -p {audit_policy_dir}")
policy_config = cluster.inventory['services']['audit'].get('cluster_policy')

if policy_config:
policy_config_file = yaml.dump(policy_config)
utils.dump_file(cluster, policy_config_file, 'audit-policy.yaml')
cluster.nodes['master'].put(io.StringIO(policy_config_file), audit_file_name, sudo=True, backup=True)
for master in cluster.nodes['master'].get_ordered_members_list():
config_new = (kubernetes.get_kubeadm_config(cluster.inventory))
master.put(io.StringIO(config_new), '/etc/kubernetes/audit-on-config.yaml', sudo=True)
master.sudo("kubeadm init phase control-plane apiserver --config=/etc/kubernetes/audit-on-config.yaml ")

else:
cluster.log.debug("Audit cluster policy config is empty, nothing will be configured ")

cluster.nodes['master'].call(utils.wait_command_successful,
command="kubectl delete pod -n kube-system "
"$(sudo kubectl get pod -n kube-system "
"| grep 'kube-apiserver' | awk '{ print $1 }')")
cluster.nodes['master'].call(utils.wait_command_successful, command="kubectl get pod -A")

def system_prepare_dns_hostname(cluster):
with RemoteExecutor(cluster):
Expand Down Expand Up @@ -490,8 +462,7 @@ def overview(cluster):
"sysctl": system_prepare_system_sysctl,
"audit": {
"install": system_install_audit,
"configure_daemon": system_prepare_audit_daemon,
"configure_policy": system_prepare_policy
"configure": system_prepare_audit
}
},
"cri": {
Expand Down
38 changes: 1 addition & 37 deletions kubemarine/resources/configurations/defaults.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,22 +27,10 @@ services:
extraArgs:
enable-admission-plugins: NodeRestriction
profiling: "false"
audit-log-path: /var/log/kubernetes/audit/audit.log
audit-policy-file: /etc/kubernetes/audit-policy.yaml
audit-log-path: /var/log/audit.log
audit-log-maxage: "30"
audit-log-maxbackup: "10"
audit-log-maxsize: "100"
extraVolumes:
- name: audit
hostPath: '{{ services["kubeadm"]["apiServer"]["extraArgs"]["audit-policy-file"] }}'
mountPath: '{{ services["kubeadm"]["apiServer"]["extraArgs"]["audit-policy-file"] }}'
readOnly: True
pathType: File
- name: audit-log
hostPath: '{% set path = services["kubeadm"]["apiServer"]["extraArgs"]["audit-log-path"].split("/") %}{{"/" + path[1] + "/" + path[2] + "/" + path[3] + "/"}}'
mountPath: '{% set path = services["kubeadm"]["apiServer"]["extraArgs"]["audit-log-path"].split("/") %}{{"/" + path[1] + "/" + path[2] + "/" + path[3] + "/"}}'
readOnly: False
pathType: DirectoryOrCreate
scheduler:
extraArgs:
profiling: "false"
Expand Down Expand Up @@ -144,30 +132,6 @@ services:
- '{% if not nodes[0]["internal_address"]|isipv4 %}localhost.localdomain{% endif %}'

audit:
cluster_policy:
apiVersion: audit.k8s.io/v1
kind: Policy
omitStages:
- "RequestReceived"
rules:
- level: Request
resources:
- group: "" # core
- group: "admissionregistration.k8s.io"
- group: "apps"
- group: "authentication.k8s.io"
- group: "authorization.k8s.io"
- group: "autoscaling"
- group: "batch"
- group: "certificates.k8s.io"
- group: "extensions"
- group: "networking.k8s.io"
- group: "policy"
- group: "rbac.authorization.k8s.io"
- group: "settings.k8s.io"
- group: "storage.k8s.io"


rules:
- '{% if services["cri"]["containerRuntime"] == "docker" %}-w /var/lib/docker -k docker{% endif %}'
- '{% if services["cri"]["containerRuntime"] == "docker" %}-w /etc/docker -k docker{% endif %}'
Expand Down

0 comments on commit 984dbdc

Please sign in to comment.