invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable #66

firatalkis · 2024-03-26T05:53:36Z

Hello,

When I have started icinga-kubernetes service on rhel8 then getting this error:

icinga-kubernetes[102140]: F0325 14:10:55.837316 102140 main.go:32] invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

Could you please help me for what did I do wrong?
Thanks.

razvanmac95 · 2024-03-26T06:35:07Z

Hi @firatalkis,

I had this issue too, and it was because I didn't mount the volume with :/.kube/config properly.
Can you check that?

Thanks,
Razvan

firatalkis · 2024-04-02T12:01:42Z

Hi @razvanmac95,

I have added oc package and ran oc login command for create /.kube/config file. Then file was created you can see below but I am getting the same error:

lippserd · 2024-04-03T12:51:19Z

Hi @firatalkis,

The KUBERNETES_MASTER environment variable expects the API endpoint of a Kubernetes master.

Please try setting KUBECONFIG instead, i.e.

export KUBECONFIG=/var/spool/icinga2/.kube/config

Best regards,
Eric

firatalkis · 2024-04-04T07:06:05Z

Hi @lippserd,

yes its worked for this issue: "try setting KUBERNETES_MASTER environment variable"

but afret that I got this error:

Actually I also added those commands to mysql configuration file

I0404 08:56:35.828377 72217 database.go:285] "Connecting to database" logger="database"
I0404 08:56:35.831957 72217 main.go:75] "Importing schema" logger="database"
F0404 08:56:35.841707 72217 main.go:80] Error 1709 (HY000): Index column size too large. The maximum column size is 767 bytes.

lippserd · 2024-04-04T07:17:05Z

Hi @firatalkis,

You also have to add innodb_default_row_format=dynamic to the mysqld configuration.

Best regards,
Eric

firatalkis · 2024-04-04T09:08:58Z

Hi @lippserd,

thanks for your help but the problems do not end for me :)

when I run "systemctl start icinga-kubernetes" then getting error you can see below:

icinga-kubernetes[127617]: F0404 11:43:43.675125 127617 main.go:32] invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
systemd[1]: Received SIGCHLD from PID 127617 (icinga-kubernet).
systemd[1]: Child 127617 (icinga-kubernet) died (code=exited, status=255/n/a)
systemd[1]: Child 127617 belongs to icinga-kubernetes.service
systemd[1]: Unwatching 127617.
systemd[1]: icinga-kubernetes.service: main process exited, code=exited, status=255/n/a
systemd[1]: icinga-kubernetes.service changed start -> failed
systemd[1]: Job icinga-kubernetes.service/start finished, result=failed
systemd[1]: Failed to start Icinga for Kubernetes.
-- Subject: Unit icinga-kubernetes.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel

-- Unit icinga-kubernetes.service has failed.

-- The result is failed.

but I run "/usr/sbin/icinga-kubernetes --config /etc/icinga-kubernetes/config.yml" this command then daemon is starting but why icinga-kubernetes want to reaching to github? our servers does not access to internet.

goroutine 332 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic({0x18ed6c0, 0xc001902dc8})
k8s.io/apimachinery@v0.29.2/pkg/util/runtime/runtime.go:75 +0x85
k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc000102540?})
k8s.io/apimachinery@v0.29.2/pkg/util/runtime/runtime.go:49 +0x6b
panic({0x18ed6c0?, 0xc001902dc8?})
runtime/panic.go:770 +0x132
github.com/icinga/icinga-kubernetes/pkg/schema/v1.(*Node).Obtain(0xc00179b440, {0x1d41d38, 0xc00190a008})
github.com/icinga/icinga-kubernetes/pkg/schema/v1/node.go:67 +0x11f9
github.com/icinga/icinga-kubernetes/pkg/sync/v1.(*Sync).sync.func1(0xc00045e6c0)
github.com/icinga/icinga-kubernetes/pkg/sync/v1/sync.go:85 +0x4f
github.com/icinga/icinga-kubernetes/pkg/sync.(*Sink).Upsert(0x2a443d0?, {0x1d2cc10, 0xc000391270}, 0xc0004f6e18?)
github.com/icinga/icinga-kubernetes/pkg/sync/sink.go:59 +0x38
github.com/icinga/icinga-kubernetes/pkg/sync.(*Controller).stream(0xc00043d5c0, {0x1d2cc10, 0xc000391270}, 0xc0001215f0)
github.com/icinga/icinga-kubernetes/pkg/sync/controller.go:93 +0x468
github.com/icinga/icinga-kubernetes/pkg/sync.(*Controller).Stream(0xc00043d5c0, {0x1d2cc10, 0xc000391270}, 0xc0001215f0)
github.com/icinga/icinga-kubernetes/pkg/sync/controller.go:55 +0x369
github.com/icinga/icinga-kubernetes/pkg/sync/v1.(*Sync).sync.func3()
github.com/icinga/icinga-kubernetes/pkg/sync/v1/sync.go:98 +0x46
golang.org/x/sync/errgroup.(*Group).Go.func1()
golang.org/x/sync@v0.6.0/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 36
golang.org/x/sync@v0.6.0/errgroup/errgroup.go:75 +0x96
panic: failed to parse CIDR : invalid CIDR address: [recovered]
panic: failed to parse CIDR : invalid CIDR address:

Apart from this I have one more question.

icigna-kubernetes using the /.kube/config for authentication to kubernetes but the duration of the token in the config is 24 hours. It seems like the token here needs to be constantly updated.

lippserd · 2024-04-12T11:57:33Z

Hi @firatalkis,

Sorry for replying a bit late. Could you please pull the image and try again? The issue should be fixed.

but I run "/usr/sbin/icinga-kubernetes --config /etc/icinga-kubernetes/config.yml" this command then daemon is starting but why icinga-kubernetes want to reaching to github? our servers does not access to internet.

Icinga for Kubernetes is not trying to reach the internet. This is a stack trace which lists all open functions involved that lead to the error. The URLs here just tell from which Go module the code is coming from.

icigna-kubernetes using the /.kube/config for authentication to kubernetes but the duration of the token in the config is 24 hours. It seems like the token here needs to be constantly updated.

There are a lot of authentication and authorization methods in K8s. I think you could just configure and use something else for Icinga for Kubernetes.

Best regards,
Eric

firatalkis · 2024-04-25T05:42:47Z

Hi @lippserd,

I am sorry for the late response because our icinga test envoriment included very old components, for example mysql 5.5 version.

so I have refresh install icinga on new virtual server as rhel8 then when I try install icinga-kubernetes got error in the below:

when I run "systemctl enable --now icinga-kubernetes" command getting same error "

then I try "/usr/sbin/icinga-kubernetes --config /etc/icinga-kubernetes/config.yml" then I see error in mariadb log file :

2024-04-25 8:41:34 587 [Warning] Aborted connection 587 to db: 'kubernetes' user: 'kubernetes' host: 'localhost' (Got an error reading communication packets)
2024-04-25 8:41:34 583 [Warning] Aborted connection 583 to db: 'kubernetes' user: 'kubernetes' host: 'localhost' (Got an error reading communication packets)
2024-04-25 8:41:34 589 [Warning] Aborted connection 589 to db: 'kubernetes' user: 'kubernetes' host: 'localhost' (Got an error reading communication packets)
2024-04-25 8:41:34 586 [Warning] Aborted connection 586 to db: 'kubernetes' user: 'kubernetes' host: 'localhost' (Got an error reading communication packets)
2024-04-25 8:41:34 584 [Warning] Aborted connection 584 to db: 'kubernetes' user: 'kubernetes' host: 'localhost' (Got an error reading communication packets)

tectumopticum · 2024-04-25T09:02:15Z

Similar here:
Required settings in /etc/icinga-kubernetes/config.yml have been made (and validated in the icingaweb2-resources-settings).
But starting the local icinga-kubernetes.service fails:

systemctl start icinga-kubernetes.service
Job for icinga-kubernetes.service failed because the control process exited with error code.
See "systemctl status icinga-kubernetes.service" and "journalctl -xeu icinga-kubernetes.service" for details.

░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit icinga-kubernetes.service has begun execution.
░░
░░ The job identifier is 4574268.
Apr 25 09:45:17 <icingahost> icinga-kubernetes[24056]: F0425 09:45:17.036354   24056 main.go:32] invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
Apr 25 09:45:17 <icingahost> systemd[1]: icinga-kubernetes.service: Main process exited, code=exited, status=255/EXCEPTION
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ An ExecStart= process belonging to unit icinga-kubernetes.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 255.
Apr 25 09:45:17 <icingahost> systemd[1]: icinga-kubernetes.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit icinga-kubernetes.service has entered the 'failed' state with result 'exit-code'.
Apr 25 09:45:17 <icingahost> systemd[1]: Failed to start Icinga for Kubernetes.
░░ Subject: A start job for unit icinga-kubernetes.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit icinga-kubernetes.service has finished with a failure.
░░
░░ The job identifier is 4574268 and the job result is failed.
Apr 25 10:08:41 <icingahost> systemd[1]: Starting Icinga for Kubernetes...
░░ Subject: A start job for unit icinga-kubernetes.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit icinga-kubernetes.service has begun execution.
░░
░░ The job identifier is 4577646.
Apr 25 10:08:41 <icingahost> icinga-kubernetes[5773]: F0425 10:08:41.708220    5773 main.go:32] invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable
Apr 25 10:08:41 <icingahost> systemd[1]: icinga-kubernetes.service: Main process exited, code=exited, status=255/EXCEPTION
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ An ExecStart= process belonging to unit icinga-kubernetes.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 255.
Apr 25 10:08:41 <icingahost> systemd[1]: icinga-kubernetes.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit icinga-kubernetes.service has entered the 'failed' state with result 'exit-code'.
Apr 25 10:08:41 <icingahost> systemd[1]: Failed to start Icinga for Kubernetes.
░░ Subject: A start job for unit icinga-kubernetes.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit icinga-kubernetes.service has finished with a failure.
░░
░░ The job identifier is 4577646 and the job result is failed.


systemctl status icinga-kubernetes.service
× icinga-kubernetes.service - Icinga for Kubernetes
     Loaded: loaded (/usr/lib/systemd/system/icinga-kubernetes.service; enabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Thu 2024-04-25 10:08:41 CEST; 26min ago
    Process: 5773 ExecStart=/usr/sbin/icinga-kubernetes --config /etc/icinga-kubernetes/config.yml (code=exited, status=255/EXCEPTION)
   Main PID: 5773 (code=exited, status=255/EXCEPTION)

The required file is available/readable:
-rw-r----- 1 icinga-kubernetes root 611 Apr 25 09:47 /etc/icinga-kubernetes/config.yml

Any additional file(s) required? Unfortunately the documentation is missing further steps how to configure a connection to kubernetes located outside the icinga-master and what to do inside kubernetes.

OS: SuSE Linux Enterprise Server SLES 15 SP5
Service-Version: 0.1.0
Mariadb-version (located on separate server): 10.6.15

lippserd · 2024-05-28T05:53:03Z

@tectumopticum You need a kubeconfig file that defines how and which Kubernetes cluster to access.

@firatalkis I'm sorry but we only support MySQL from version 8 and MariaDB from version 10.4. However, the documentation states something else that needs to be fixed. Maybe it's also worth a try to increase max_allowed_packet.

lippserd · 2024-05-28T05:59:09Z

@Jan-Schuppik Please fix the documentation regarding the supported database versions and create a section explaining how to use the kubeconfig file and what it's for. You can link to this section where we write

With locally accessible kubeconfig ...

firatalkis · 2024-05-29T10:50:13Z

Hi @lippserd,

our mariadb versiyon:

Do I have upgrade to 10.4 version for fixing below issue?

I have changed max_allowed_packet value from 512 MB to 2G. After that I fresh install icinga-kubernetes then when I run service with systemctl got permission error as "status=255" and I changed permission this folder "/etc/icinga-kubernetes" to 755 than this issue is fixed.

May 28 17:30:21 host systemd[1]: icinga-kubernetes.service: Main process exited, code=exited, status=255/n/a
May 28 17:30:21 host systemd[1]: icinga-kubernetes.service: Failed with result 'exit-code'.
May 28 17:30:21 host systemd[1]: Failed to start Icinga for Kubernetes.

I used "xx" service account for create "/var/spool/icinga2/.kube/config" file with "oc login" command but icinga-kubernetes trying to collect data from kubernetes with "system:anonymous" user and that user getting permission error:

May 29 11:55:52 host icinga-kubernetes[813204]: E0529 11:55:52.185682 813204 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch *v1.Ingress: failed to list *v1.Ingress: ingresses.networking.k8s.io is forbidden: User "system:anonymous" cannot list resource "ingresses" in API group "networking.k8s.io" at the cluster scope
May 29 11:55:57 host icinga-kubernetes[813204]: W0529 11:55:57.188885 813204 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list *v1.Job: jobs.batch is forbidden: User "system:anonymous" cannot list resource "jobs" in API group "batch" at the cluster scope
May 29 11:55:57 host icinga-kubernetes[813204]: E0529 11:55:57.188927 813204 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch *v1.Job: failed to list *v1.Job: jobs.batch is forbidden: User "system:anonymous" cannot list resource "jobs" in API group "batch" at the cluster scope
May 29 11:55:58 host icinga-kubernetes[813204]: W0529 11:55:58.399065 813204 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list *v1.Node: nodes is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
May 29 11:55:58 host icinga-kubernetes[813204]: E0529 11:55:58.399106 813204 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:anonymous" cannot list resource "nodes" in API group "" at the cluster scope
May 29 11:55:58 host icinga-kubernetes[813204]: W0529 11:55:58.845428 813204 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list *v1.Deployment: deployments.apps is forbidden: User "system:anonymous" cannot list resource "deployments" in API group "apps" at the cluster scope
May 29 11:55:58 host icinga-kubernetes[813204]: E0529 11:55:58.845475 813204 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch *v1.Deployment: failed to list *v1.Deployment: deployments.apps is forbidden: User "system:anonymous" cannot list resource "deployments" in API group "apps" at the cluster scope
May 29 11:55:58 host icinga-kubernetes[813204]: W0529 11:55:58.881140 813204 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:anonymous" cannot list resource "statefulsets" in API group "apps" at the cluster scope
May 29 11:55:58 host icinga-kubernetes[813204]: E0529 11:55:58.881180 813204 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch *v1.StatefulSet: failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:anonymous" cannot list resource "statefulsets" in API group "apps" at the cluster scope
May 29 11:55:59 host icinga-kubernetes[813204]: W0529 11:55:59.647303 813204 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list *v1.Event: events.events.k8s.io is forbidden: User "system:anonymous" cannot list resource "events" in API group "events.k8s.io" at the cluster scope
May 29 11:55:59 host icinga-kubernetes[813204]: E0529 11:55:59.647349 813204 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch *v1.Event: failed to list *v1.Event: events.events.k8s.io is forbidden: User "system:anonymous" cannot list resource "events" in API group "events.k8s.io" at the cluster scope
May 29 11:56:00 host icinga-kubernetes[813204]: W0529 11:56:00.667714 813204 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:anonymous" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope

lippserd assigned Jan-Schuppik May 28, 2024

Jan-Schuppik mentioned this issue May 29, 2024

Fix documentation #94

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable #66

invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable #66

firatalkis commented Mar 26, 2024

razvanmac95 commented Mar 26, 2024

firatalkis commented Apr 2, 2024

lippserd commented Apr 3, 2024

firatalkis commented Apr 4, 2024

lippserd commented Apr 4, 2024

firatalkis commented Apr 4, 2024

lippserd commented Apr 12, 2024

firatalkis commented Apr 25, 2024

tectumopticum commented Apr 25, 2024

lippserd commented May 28, 2024 •

edited

Loading

lippserd commented May 28, 2024

firatalkis commented May 29, 2024

invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable #66

invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable #66

Comments

firatalkis commented Mar 26, 2024

razvanmac95 commented Mar 26, 2024

firatalkis commented Apr 2, 2024

lippserd commented Apr 3, 2024

firatalkis commented Apr 4, 2024

lippserd commented Apr 4, 2024

firatalkis commented Apr 4, 2024

-- Unit icinga-kubernetes.service has failed.

lippserd commented Apr 12, 2024

firatalkis commented Apr 25, 2024

tectumopticum commented Apr 25, 2024

lippserd commented May 28, 2024 • edited Loading

lippserd commented May 28, 2024

firatalkis commented May 29, 2024

lippserd commented May 28, 2024 •

edited

Loading