Cannot apply deployment with error "Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'"

[ibrahim-osama-amin]

Hello guys,

Hope you are doing great.
This is my first post here, so I hope I am doing things correctly. I checked previous issues and I did not find this one.

SUMMARY

The first task works normally. The second task gets stuck with this weird error:

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'
fatal: [192.168.111.138]: FAILED! => {"changed": false, "msg": "Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'"}

It is weird because I am running as root so I shouldn't face any permission issues.

The ansible target user is kube and it has all the requirements installed:

[kube@localhost ~]$ python3 -c "import openshift"
[kube@localhost ~]$ python3 -c "import yaml"
[kube@localhost ~]$ python3 -c "import jsonpatch"

I tried to work with community.kubernetes.k8s and same error
Also, I tried to play around with the names of the nginx deployment and it is the same issue.
This is my repo if you want to check and take a look at the history:
https://github.com/ibrahim-osama-amin/Ansible

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.kubernetes.k8s and kubernetes.core.k8s

ANSIBLE VERSION

ansible [core 2.14.14]
  config file = /root/ansible/Ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.18 (main, May 16 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

[root@localhost Ansible]# ansible-config dump --only-changed
CONFIG_FILE() = /root/ansible/Ansible/ansible.cfg
DEFAULT_HOST_LIST(/root/ansible/Ansible/ansible.cfg) = ['/root/ansible/Ansible/hosts']
HOST_KEY_CHECKING(/root/ansible/Ansible/ansible.cfg) = False

CONFIGURATION

[root@localhost Ansible]# ansible-config dump --only-changed
CONFIG_FILE() = /root/ansible/Ansible/ansible.cfg
DEFAULT_HOST_LIST(/root/ansible/Ansible/ansible.cfg) = ['/root/ansible/Ansible/hosts']
HOST_KEY_CHECKING(/root/ansible/Ansible/ansible.cfg) = False

OS / ENVIRONMENT

Ansible host:

[root@localhost Ansible]# hostnamectl
   Static hostname: (unset)
Transient hostname: localhost
         Icon name: computer-vm
           Chassis: vm 🖴
        Machine ID: 67bcc410514841869f3234ef45756747
           Boot ID: 0532904d3785438ab6526e10c1d1e147
    Virtualization: vmware
  Operating System: Red Hat Enterprise Linux 9.4 (Plow)
       CPE OS Name: cpe:/o:redhat:enterprise_linux:9::baseos
            Kernel: Linux 5.14.0-427.13.1.el9_4.x86_64
      Architecture: x86-64
   Hardware Vendor: VMware, Inc.
    Hardware Model: VMware 

Ansible target:

   Static hostname: (unset)
Transient hostname: localhost
         Icon name: computer-vm
           Chassis: vm 🖴
        Machine ID: 1ac6a73bd74247f898bfa530e6d526c8
           Boot ID: 3aa46e091ef14b4da82473701a7daa95
    Virtualization: vmware
  Operating System: Red Hat Enterprise Linux 9.3 (Plow)
       CPE OS Name: cpe:/o:redhat:enterprise_linux:9::baseos
            Kernel: Linux 5.14.0-362.8.1.el9_3.x86_64
      Architecture: x86-64
   Hardware Vendor: VMware, Inc.
    Hardware Model: VMware Virtual Platform
  Firmware Version: 6.00

STEPS TO REPRODUCE

Create two Redhat VMs, install requirements on the source and the destination.
Run kubectl on the destination with user kube

---
- name: Deploy app in a new namespace
  hosts: local
  tasks:
    - name: Ensure my-app namespace is present 
      community.kubernetes.k8s:
        name: my-app
        api_version: v1
        kind: Namespace
        state: present  

    - name: Ensure nginx deployment with its service are running
      community.kubernetes.k8s:
        src: /root/ansible/Ansible/k8s_deployments/nginx.yaml
        state: present
        kubeconfig: /home/kube/.kube/config
        namespace: my-app

EXPECTED RESULTS

I should see a pod created and a service created on the target.

ACTUAL RESULTS

TASK [Ensure nginx deployment with its service are running] **********************************************************************************************************************************************
task path: /root/ansible/Ansible/deploy-to-k8s.yaml:12
redirecting (type: modules) community.kubernetes.k8s to kubernetes.core.k8s
redirecting (type: action) community.kubernetes.k8s to kubernetes.core.k8s_info
redirecting (type: action) community.kubernetes.k8s to kubernetes.core.k8s_info
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' 192.168.111.138 '/bin/sh -c '"'"'echo ~kube && sleep 0'"'"''
<192.168.111.138> (0, b'/home/kube\n', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' 192.168.111.138 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/kube/.ansible/tmp `"&& mkdir "` echo /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893 `" && echo ansible-tmp-1723209062.7628653-4397-109600890829893="` echo /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893 `" ) && sleep 0'"'"''
<192.168.111.138> (0, b'ansible-tmp-1723209062.7628653-4397-109600890829893=/home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893\n', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
redirecting (type: modules) community.kubernetes.k8s to kubernetes.core.k8s
Using module file /root/.ansible/collections/ansible_collections/kubernetes/core/plugins/modules/k8s.py
<192.168.111.138> PUT /root/.ansible/tmp/ansible-local-4367vfq387sm/tmpij2nghdd TO /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py
<192.168.111.138> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' '[192.168.111.138]'
<192.168.111.138> (0, b'sftp> put /root/.ansible/tmp/ansible-local-4367vfq387sm/tmpij2nghdd /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py\n', b'OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for \'final all\' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched \'final\'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for \'final all\' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched \'final\'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts\' -> \'/root/.ssh/known_hosts\'\r\ndebug3: expanded UserKnownHostsFile \'~/.ssh/known_hosts2\' -> \'/root/.ssh/known_hosts2\'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug2: Remote version: 3\r\ndebug2: Server supports extension "posix-rename@openssh.com" revision 1\r\ndebug2: Server supports extension "statvfs@openssh.com" revision 2\r\ndebug2: Server supports extension "fstatvfs@openssh.com" revision 2\r\ndebug2: Server supports extension "hardlink@openssh.com" revision 1\r\ndebug2: Server supports extension "fsync@openssh.com" revision 1\r\ndebug2: Server supports extension "lsetstat@openssh.com" revision 1\r\ndebug2: Server supports extension "limits@openssh.com" revision 1\r\ndebug2: Server supports extension "expand-path@openssh.com" revision 1\r\ndebug3: Sent message limits@openssh.com I:1\r\ndebug3: Received limits reply T:201 I:1\r\ndebug1: Using server download size 261120\r\ndebug1: Using server upload size 261120\r\ndebug1: Server handle limit 1019; using 64\r\ndebug3: Sent message fd 3 T:16 I:2\r\ndebug3: SSH2_FXP_REALPATH . -> /home/kube\r\ndebug3: Looking up /root/.ansible/tmp/ansible-local-4367vfq387sm/tmpij2nghdd\r\ndebug3: Sent message fd 3 T:17 I:3\r\ndebug1: Couldn\'t stat remote file: No such file or directory\r\ndebug3: Sent dest message SSH2_FXP_OPEN I:4 P:/home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py M:0x001a\r\ndebug3: Sent message SSH2_FXP_WRITE I:6 O:0 S:235207\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: In write loop, ack for 6 235207 bytes at 0\r\ndebug3: Sent message SSH2_FXP_CLOSE I:5\r\ndebug3: SSH2_FXP_STATUS 0\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n')
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' 192.168.111.138 '/bin/sh -c '"'"'chmod u+x /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/ /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py && sleep 0'"'"''
<192.168.111.138> (0, b'', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' -tt 192.168.111.138 '/bin/sh -c '"'"'/usr/bin/python3 /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/AnsiballZ_k8s.py && sleep 0'"'"''
<192.168.111.138> (1, b'\r\n{"exception": "Traceback (most recent call last):\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py\\", line 49, in get_definitions\\n    definitions = create_definitions(params)\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/resource.py\\", line 61, in create_definitions\\n    definitions = from_file(d)\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/resource.py\\", line 98, in from_file\\n    with open(path, \\"rb\\") as f:\\nPermissionError: [Errno 13] Permission denied: \'/root/ansible/Ansible/k8s_deployments/nginx.yaml\'\\n\\nThe above exception was the direct cause of the following exception:\\n\\nTraceback (most recent call last):\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py\\", line 503, in main\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py\\", line 91, in run_module\\n    definitions = get_definitions(svc, module.params)\\n  File \\"/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py\\", line 52, in get_definitions\\n    raise CoreException(msg) from e\\nansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Failed to load resource definition: [Errno 13] Permission denied: \'/root/ansible/Ansible/k8s_deployments/nginx.yaml\'\\n", "failed": true, "msg": "Failed to load resource definition: [Errno 13] Permission denied: \'/root/ansible/Ansible/k8s_deployments/nginx.yaml\'", "invocation": {"module_args": {"src": "/root/ansible/Ansible/k8s_deployments/nginx.yaml", "state": "present", "kubeconfig": "/home/kube/.kube/config", "namespace": "my-app", "api_version": "v1", "wait": false, "wait_sleep": 5, "wait_timeout": 120, "append_hash": false, "apply": false, "continue_on_error": false, "force": false, "delete_all": false, "kind": null, "name": null, "resource_definition": null, "context": null, "host": null, "api_key": null, "username": null, "password": null, "validate_certs": null, "ca_cert": null, "client_cert": null, "client_key": null, "proxy": null, "no_proxy": null, "proxy_headers": null, "persist_config": null, "impersonate_user": null, "impersonate_groups": null, "wait_condition": null, "merge_type": null, "validate": null, "template": null, "delete_options": null, "label_selectors": null, "generate_name": null, "server_side_apply": null, "hidden_fields": null}}}\r\n', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to 192.168.111.138 closed.\r\n")
<192.168.111.138> Failed to connect to the host via ssh: OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'
debug2: match not found
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
debug1: configuration requests final Match pass
debug2: resolve_canonicalize: hostname 192.168.111.138 is address
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'
debug2: match found
debug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]
debug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'
debug1: auto-mux: Trying existing master
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 4378
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 1
Shared connection to 192.168.111.138 closed.
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube
<192.168.111.138> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/root/.ssh/kube"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kube"' -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/8e11ae4af7"' 192.168.111.138 '/bin/sh -c '"'"'rm -f -r /home/kube/.ansible/tmp/ansible-tmp-1723209062.7628653-4397-109600890829893/ > /dev/null 2>&1 && sleep 0'"'"''
<192.168.111.138> (0, b'', b"OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug2: resolve_canonicalize: hostname 192.168.111.138 is address\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host 192.168.111.138 originally 192.168.111.138\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 4378\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\n")
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 49, in get_definitions
    definitions = create_definitions(params)
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/resource.py", line 61, in create_definitions
    definitions = from_file(d)
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/resource.py", line 98, in from_file
    with open(path, "rb") as f:
PermissionError: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/modules/k8s.py", line 503, in main
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 91, in run_module
    definitions = get_definitions(svc, module.params)
  File "/tmp/ansible_community.kubernetes.k8s_payload_q_jzlnzh/ansible_community.kubernetes.k8s_payload.zip/ansible_collections/kubernetes/core/plugins/module_utils/k8s/runner.py", line 52, in get_definitions
    raise CoreException(msg) from e
ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'
fatal: [192.168.111.138]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_key": null,
            "api_version": "v1",
            "append_hash": false,
            "apply": false,
            "ca_cert": null,
            "client_cert": null,
            "client_key": null,
            "context": null,
            "continue_on_error": false,
            "delete_all": false,
            "delete_options": null,
            "force": false,
            "generate_name": null,
            "hidden_fields": null,
            "host": null,
            "impersonate_groups": null,
            "impersonate_user": null,
            "kind": null,
            "kubeconfig": "/home/kube/.kube/config",
            "label_selectors": null,
            "merge_type": null,
            "name": null,
            "namespace": "my-app",
            "no_proxy": null,
            "password": null,
            "persist_config": null,
            "proxy": null,
            "proxy_headers": null,
            "resource_definition": null,
            "server_side_apply": null,
            "src": "/root/ansible/Ansible/k8s_deployments/nginx.yaml",
            "state": "present",
            "template": null,
            "username": null,
            "validate": null,
            "validate_certs": null,
            "wait": false,
            "wait_condition": null,
            "wait_sleep": 5,
            "wait_timeout": 120
        }
    },
    "msg": "Failed to load resource definition: [Errno 13] Permission denied: '/root/ansible/Ansible/k8s_deployments/nginx.yaml'"
}

[ibrahim-osama-amin]

Tried to upgrade ansible today but I am still getting the same issue, most likely I am doing something incorrectly and this is not a bug.

[root@localhost Ansible]# ansible --version
ansible [core 2.17.0]
  config file = /root/ansible/Ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.12/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.12.1 (main, May  3 2024, 00:00:00) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

[dkarpele]

Hi @ibrahim-osama-amin

It is weird because I am running as root so I shouldn't face any permission issues.

There is a line in log:
<192.168.111.138> ESTABLISH SSH CONNECTION FOR USER: kube

Are you sure that you are running as root?
You can also try to change permissions for nginx.yaml or move the file under /home/kube/. Pay attention that this file is in /root/ansible/Ansible/k8s_deployments/ .
Also you can try to experiment with become keyword

[gravesm]

As @dkarpele has pointed out, you are running as the kube user on the host where the module is getting executed. We would need to see your inventory file to provide further help.