Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to connect to proxy when setting up agent nodes #9860

Closed
pgregorio-signalytic opened this issue Apr 3, 2024 · 2 comments
Closed

Unable to connect to proxy when setting up agent nodes #9860

pgregorio-signalytic opened this issue Apr 3, 2024 · 2 comments

Comments

@pgregorio-signalytic
Copy link

Environmental Info:
K3s Version:

root@master:/home/vagrant# k3s -v
k3s version v1.28.8+k3s1 (653dd61a)
go version go1.21.8

Node(s) CPU architecture, OS, and Version:

root@master:/home/vagrant# uname -a
Linux master 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration:

1 server, 2 agents:

Vagrant.configure(2) do |config|
  config.vm.box = "generic/ubuntu2204"

  config.vm.provider "virtualbox" do |vb|
    vb.memory = "1024"
  end

  config.vm.define "master" do |master|
    master.vm.hostname = "master"
    master.vm.network "private_network", ip: "192.168.56.2"
  end

  config.vm.define "node1" do |node1|
    node1.vm.hostname = "node1"
    node1.vm.network "private_network", ip: "192.168.56.11"
  end

  config.vm.define "node2" do |node2|
    node2.vm.hostname = "node2"
    node2.vm.network "private_network", ip: "192.168.56.22"
  end
end

Describe the bug:

When setting up the agent nodes, I get the following errors:

Failed to connect to proxy. Empty dialer response
Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused

Steps To Reproduce:

  • Installed K3s with the following instructions:

K3S

Use Vagrant to easily spin multiple VMs (requires VirtualBox). First we need to install both packages:

# Install virtualbox and vagrant
sudo apt install virtualbox vagrant -y

We should now be able to create the VMs we need to test k3s:

# Run the VMs
vagrant up

This should spin up 3 Ubuntu 22.04 VMs with their respective hostnames and IP addresses as defined on the Vagrantfile.

Update firewall

For Ubuntu VMs, a few extra steps are needed:

# It is recommended to turn off ufw (uncomplicated firewall)
ufw disable

# If you wish to keep ufw enabled, by default, the following rules are required
ufw allow 6443/tcp                   #apiserver
ufw allow from 10.42.0.0/16 to any   #pods
ufw allow from 10.43.0.0/16 to any   #services

MASTER

On the master VM, install k3s:

# Install k3s as the master
curl -sfL https://get.k3s.io | sh -

# Check that the service is running
systemctl status k3s

# Check the nodes
k3s kubectl get nodes

Get the token we need to use when installing k3s on the other nodes:

cat /var/lib/rancher/k3s/server/node-token

NODE1

On the node1 VM, install k3s as an agent:

# Install k3s as an agent node
curl -sfL https://get.k3s.io | K3S_URL=https://<MASTER_IP>:6443 K3S_TOKEN=<TOKEN> sh -

NODE2

On the node2 VM, install k3s as an agent:

# Install k3s as an agent node
curl -sfL https://get.k3s.io | K3S_URL=https://<MASTER_IP>:6443 K3S_TOKEN=<TOKEN> sh -

Expected behavior:

The agent nodes should be able to connect to the master node.

Actual behavior:

The installation on both agent nodes get stuck at and eventually times out:

[INFO]  systemd: Starting k3s-agent

Additional context / logs:

● k3s-agent.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s-agent.service; enabled; vendor preset: enabled)
     Active: activating (start) since Wed 2024-04-03 16:18:27 UTC; 1min 49s ago
       Docs: https://k3s.io
    Process: 3911 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service 2>/dev/null (code=exited, status=0/SUCCESS)
    Process: 3913 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
    Process: 3914 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Main PID: 3915 (k3s-agent)
      Tasks: 16
     Memory: 251.7M
        CPU: 3.144s
     CGroup: /system.slice/k3s-agent.service
             ├─3915 "/usr/local/bin/k3s agent"
             └─3938 "containerd " "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" ""

Apr 03 16:20:08 node1 k3s[3915]: time="2024-04-03T16:20:08Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:10 node1 k3s[3915]: W0403 16:20:10.470622    3915 reflector.go:535] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:39464->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:42520->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:20:10 node1 k3s[3915]: I0403 16:20:10.470982    3915 trace.go:236] Trace[1903520160]: "Reflector ListAndWatch" name:k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229 (03-Apr-2024 16:20:00.430) (total time: 10040ms):
Apr 03 16:20:10 node1 k3s[3915]: Trace[1903520160]: ---"Objects listed" error:Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:39464->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:42520->127.0.0.1:6444: read: connection reset by peer 10039ms (16:20:10.470)
Apr 03 16:20:10 node1 k3s[3915]: Trace[1903520160]: [10.040077312s] [10.040077312s] END
Apr 03 16:20:10 node1 k3s[3915]: E0403 16:20:10.471095    3915 reflector.go:147] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:39464->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:42520->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:20:12 node1 k3s[3915]: time="2024-04-03T16:20:12Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": EOF"
Apr 03 16:20:13 node1 k3s[3915]: time="2024-04-03T16:20:13Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:20:13 node1 k3s[3915]: time="2024-04-03T16:20:13Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:13 node1 k3s[3915]: time="2024-04-03T16:20:13Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"

Apr 03 16:18:27 node1 systemd[1]: Starting Lightweight Kubernetes...
Apr 03 16:18:27 node1 sh[3911]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Apr 03 16:18:27 node1 k3s[3915]: time="2024-04-03T16:18:27Z" level=info msg="Acquiring lock file /var/lib/rancher/k3s/data/.lock"
Apr 03 16:18:27 node1 k3s[3915]: time="2024-04-03T16:18:27Z" level=info msg="Preparing data dir /var/lib/rancher/k3s/data/ef92c15b8f1c59f266ea4a9589be87b1b799d1682673f29721e8a1f7b0a4a97b"
Apr 03 16:18:30 node1 k3s[3915]: time="2024-04-03T16:18:30Z" level=info msg="Starting k3s agent v1.28.8+k3s1 (653dd61a)"
Apr 03 16:18:30 node1 k3s[3915]: time="2024-04-03T16:18:30Z" level=info msg="Adding server to load balancer k3s-agent-load-balancer: 192.168.56.2:6443"
Apr 03 16:18:30 node1 k3s[3915]: time="2024-04-03T16:18:30Z" level=info msg="Running load balancer k3s-agent-load-balancer 127.0.0.1:6444 -> [192.168.56.2:6443] [default: 192.168.56.2:6443]"
Apr 03 16:18:31 node1 k3s[3915]: time="2024-04-03T16:18:31Z" level=info msg="Module overlay was already loaded"
Apr 03 16:18:31 node1 k3s[3915]: time="2024-04-03T16:18:31Z" level=info msg="Module br_netfilter was already loaded"
Apr 03 16:18:31 node1 k3s[3915]: time="2024-04-03T16:18:31Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400"
Apr 03 16:18:31 node1 k3s[3915]: time="2024-04-03T16:18:31Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600"
Apr 03 16:18:31 node1 k3s[3915]: time="2024-04-03T16:18:31Z" level=info msg="Set sysctl 'net/ipv4/conf/all/forwarding' to 1"
Apr 03 16:18:31 node1 k3s[3915]: time="2024-04-03T16:18:31Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072"
Apr 03 16:18:31 node1 k3s[3915]: time="2024-04-03T16:18:31Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
Apr 03 16:18:31 node1 k3s[3915]: time="2024-04-03T16:18:31Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=info msg="containerd is now running"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=info msg="Getting list of apiserver endpoints from server"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=info msg="Updated load balancer k3s-agent-load-balancer default server address -> 10.0.2.15:6443"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=info msg="Adding server to load balancer k3s-agent-load-balancer: 10.0.2.15:6443"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=info msg="Removing server from load balancer k3s-agent-load-balancer: 192.168.56.2:6443"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=info msg="Updated load balancer k3s-agent-load-balancer server addresses -> [10.0.2.15:6443] [default: 10.0.2.15:6443]"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=node1 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-ip=10.0.2.15 --node-labels= --pod-infra-container-image=rancher/mirrored-pause:3.6 --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:32 node1 k3s[3915]: time="2024-04-03T16:18:32Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:38152->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:18:37 node1 k3s[3915]: time="2024-04-03T16:18:37Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:18:37 node1 k3s[3915]: time="2024-04-03T16:18:37Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:38314->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:18:37 node1 k3s[3915]: time="2024-04-03T16:18:37Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:37 node1 k3s[3915]: time="2024-04-03T16:18:37Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:42 node1 k3s[3915]: time="2024-04-03T16:18:42Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:18:42 node1 k3s[3915]: time="2024-04-03T16:18:42Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:42 node1 k3s[3915]: time="2024-04-03T16:18:42Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:42 node1 k3s[3915]: time="2024-04-03T16:18:42Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:57764->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:18:43 node1 k3s[3915]: W0403 16:18:43.148052    3915 reflector.go:535] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:57806->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:57744->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:18:43 node1 k3s[3915]: I0403 16:18:43.148308    3915 trace.go:236] Trace[1871427865]: "Reflector ListAndWatch" name:k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229 (03-Apr-2024 16:18:32.941) (total time: 10207ms):
Apr 03 16:18:43 node1 k3s[3915]: Trace[1871427865]: ---"Objects listed" error:Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:57806->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:57744->127.0.0.1:6444: read: connection reset by peer 10207ms (16:18:43.148)
Apr 03 16:18:43 node1 k3s[3915]: Trace[1871427865]: [10.207250423s] [10.207250423s] END
Apr 03 16:18:43 node1 k3s[3915]: E0403 16:18:43.148448    3915 reflector.go:147] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:57806->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:57744->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:18:47 node1 k3s[3915]: time="2024-04-03T16:18:47Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:58004->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:18:47 node1 k3s[3915]: time="2024-04-03T16:18:47Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:18:47 node1 k3s[3915]: time="2024-04-03T16:18:47Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:47 node1 k3s[3915]: time="2024-04-03T16:18:47Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:52 node1 k3s[3915]: time="2024-04-03T16:18:52Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:60786->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:18:52 node1 k3s[3915]: time="2024-04-03T16:18:52Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:18:52 node1 k3s[3915]: time="2024-04-03T16:18:52Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:52 node1 k3s[3915]: time="2024-04-03T16:18:52Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:54 node1 k3s[3915]: W0403 16:18:54.105835    3915 reflector.go:535] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": EOF - error from a previous attempt: read tcp 127.0.0.1:60788->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:18:54 node1 k3s[3915]: I0403 16:18:54.105899    3915 trace.go:236] Trace[232591820]: "Reflector ListAndWatch" name:k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229 (03-Apr-2024 16:18:43.982) (total time: 10123ms):
Apr 03 16:18:54 node1 k3s[3915]: Trace[232591820]: ---"Objects listed" error:Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": EOF - error from a previous attempt: read tcp 127.0.0.1:60788->127.0.0.1:6444: read: connection reset by peer 10122ms (16:18:54.105)
Apr 03 16:18:54 node1 k3s[3915]: Trace[232591820]: [10.123021188s] [10.123021188s] END
Apr 03 16:18:54 node1 k3s[3915]: E0403 16:18:54.105911    3915 reflector.go:147] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": EOF - error from a previous attempt: read tcp 127.0.0.1:60788->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:18:57 node1 k3s[3915]: time="2024-04-03T16:18:57Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:18:57 node1 k3s[3915]: time="2024-04-03T16:18:57Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:57 node1 k3s[3915]: time="2024-04-03T16:18:57Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:18:57 node1 k3s[3915]: time="2024-04-03T16:18:57Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:60948->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:19:02 node1 k3s[3915]: time="2024-04-03T16:19:02Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:38052->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:19:02 node1 k3s[3915]: time="2024-04-03T16:19:02Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:02 node1 k3s[3915]: time="2024-04-03T16:19:02Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:02 node1 k3s[3915]: time="2024-04-03T16:19:02Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:06 node1 k3s[3915]: W0403 16:19:06.982008    3915 reflector.go:535] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": EOF - error from a previous attempt: read tcp 127.0.0.1:38196->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:19:06 node1 k3s[3915]: I0403 16:19:06.982401    3915 trace.go:236] Trace[2059427684]: "Reflector ListAndWatch" name:k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229 (03-Apr-2024 16:18:56.861) (total time: 10120ms):
Apr 03 16:19:06 node1 k3s[3915]: Trace[2059427684]: ---"Objects listed" error:Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": EOF - error from a previous attempt: read tcp 127.0.0.1:38196->127.0.0.1:6444: read: connection reset by peer 10120ms (16:19:06.981)
Apr 03 16:19:06 node1 k3s[3915]: Trace[2059427684]: [10.120741384s] [10.120741384s] END
Apr 03 16:19:06 node1 k3s[3915]: E0403 16:19:06.982546    3915 reflector.go:147] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": EOF - error from a previous attempt: read tcp 127.0.0.1:38196->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:19:07 node1 k3s[3915]: time="2024-04-03T16:19:07Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:38292->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:19:07 node1 k3s[3915]: time="2024-04-03T16:19:07Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:07 node1 k3s[3915]: time="2024-04-03T16:19:07Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:07 node1 k3s[3915]: time="2024-04-03T16:19:07Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:12 node1 k3s[3915]: time="2024-04-03T16:19:12Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": EOF"
Apr 03 16:19:13 node1 k3s[3915]: time="2024-04-03T16:19:13Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:13 node1 k3s[3915]: time="2024-04-03T16:19:13Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:13 node1 k3s[3915]: time="2024-04-03T16:19:13Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:17 node1 k3s[3915]: time="2024-04-03T16:19:17Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:41024->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:19:18 node1 k3s[3915]: time="2024-04-03T16:19:18Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:18 node1 k3s[3915]: time="2024-04-03T16:19:18Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:18 node1 k3s[3915]: time="2024-04-03T16:19:18Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:20 node1 k3s[3915]: W0403 16:19:20.710333    3915 reflector.go:535] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:35684->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:41088->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:19:20 node1 k3s[3915]: I0403 16:19:20.710377    3915 trace.go:236] Trace[1014071780]: "Reflector ListAndWatch" name:k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229 (03-Apr-2024 16:19:10.543) (total time: 10166ms):
Apr 03 16:19:20 node1 k3s[3915]: Trace[1014071780]: ---"Objects listed" error:Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:35684->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:41088->127.0.0.1:6444: read: connection reset by peer 10166ms (16:19:20.710)
Apr 03 16:19:20 node1 k3s[3915]: Trace[1014071780]: [10.166928099s] [10.166928099s] END
Apr 03 16:19:20 node1 k3s[3915]: E0403 16:19:20.710386    3915 reflector.go:147] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:35684->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:41088->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:19:22 node1 k3s[3915]: time="2024-04-03T16:19:22Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:35732->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:19:23 node1 k3s[3915]: time="2024-04-03T16:19:23Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:23 node1 k3s[3915]: time="2024-04-03T16:19:23Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:23 node1 k3s[3915]: time="2024-04-03T16:19:23Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:27 node1 k3s[3915]: time="2024-04-03T16:19:27Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:35860->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:19:28 node1 k3s[3915]: time="2024-04-03T16:19:28Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:28 node1 k3s[3915]: time="2024-04-03T16:19:28Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:28 node1 k3s[3915]: time="2024-04-03T16:19:28Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:32 node1 k3s[3915]: time="2024-04-03T16:19:32Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": EOF"
Apr 03 16:19:33 node1 k3s[3915]: time="2024-04-03T16:19:33Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:33 node1 k3s[3915]: time="2024-04-03T16:19:33Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:33 node1 k3s[3915]: time="2024-04-03T16:19:33Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:37 node1 k3s[3915]: time="2024-04-03T16:19:37Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": EOF"
Apr 03 16:19:38 node1 k3s[3915]: time="2024-04-03T16:19:38Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:38 node1 k3s[3915]: time="2024-04-03T16:19:38Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:38 node1 k3s[3915]: time="2024-04-03T16:19:38Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:41 node1 k3s[3915]: W0403 16:19:41.276237    3915 reflector.go:535] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:51740->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:51712->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:19:41 node1 k3s[3915]: I0403 16:19:41.276282    3915 trace.go:236] Trace[788827548]: "Reflector ListAndWatch" name:k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229 (03-Apr-2024 16:19:31.151) (total time: 10125ms):
Apr 03 16:19:41 node1 k3s[3915]: Trace[788827548]: ---"Objects listed" error:Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:51740->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:51712->127.0.0.1:6444: read: connection reset by peer 10125ms (16:19:41.276)
Apr 03 16:19:41 node1 k3s[3915]: Trace[788827548]: [10.125206812s] [10.125206812s] END
Apr 03 16:19:41 node1 k3s[3915]: E0403 16:19:41.276292    3915 reflector.go:147] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:51740->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:51712->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:19:42 node1 k3s[3915]: time="2024-04-03T16:19:42Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:51798->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:19:43 node1 k3s[3915]: time="2024-04-03T16:19:43Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:43 node1 k3s[3915]: time="2024-04-03T16:19:43Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:43 node1 k3s[3915]: time="2024-04-03T16:19:43Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:47 node1 k3s[3915]: time="2024-04-03T16:19:47Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": EOF"
Apr 03 16:19:48 node1 k3s[3915]: time="2024-04-03T16:19:48Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:48 node1 k3s[3915]: time="2024-04-03T16:19:48Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:48 node1 k3s[3915]: time="2024-04-03T16:19:48Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:52 node1 k3s[3915]: time="2024-04-03T16:19:52Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": EOF"
Apr 03 16:19:53 node1 k3s[3915]: time="2024-04-03T16:19:53Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:53 node1 k3s[3915]: time="2024-04-03T16:19:53Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:53 node1 k3s[3915]: time="2024-04-03T16:19:53Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:57 node1 k3s[3915]: time="2024-04-03T16:19:57Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:37044->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:19:58 node1 k3s[3915]: time="2024-04-03T16:19:58Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:19:58 node1 k3s[3915]: time="2024-04-03T16:19:58Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:19:58 node1 k3s[3915]: time="2024-04-03T16:19:58Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:02 node1 k3s[3915]: time="2024-04-03T16:20:02Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:42192->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:20:03 node1 k3s[3915]: time="2024-04-03T16:20:03Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:20:03 node1 k3s[3915]: time="2024-04-03T16:20:03Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:03 node1 k3s[3915]: time="2024-04-03T16:20:03Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:07 node1 k3s[3915]: time="2024-04-03T16:20:07Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:42448->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:20:08 node1 k3s[3915]: time="2024-04-03T16:20:08Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:20:08 node1 k3s[3915]: time="2024-04-03T16:20:08Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:08 node1 k3s[3915]: time="2024-04-03T16:20:08Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:10 node1 k3s[3915]: W0403 16:20:10.470622    3915 reflector.go:535] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:39464->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:42520->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:20:10 node1 k3s[3915]: I0403 16:20:10.470982    3915 trace.go:236] Trace[1903520160]: "Reflector ListAndWatch" name:k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229 (03-Apr-2024 16:20:00.430) (total time: 10040ms):
Apr 03 16:20:10 node1 k3s[3915]: Trace[1903520160]: ---"Objects listed" error:Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:39464->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:42520->127.0.0.1:6444: read: connection reset by peer 10039ms (16:20:10.470)
Apr 03 16:20:10 node1 k3s[3915]: Trace[1903520160]: [10.040077312s] [10.040077312s] END
Apr 03 16:20:10 node1 k3s[3915]: E0403 16:20:10.471095    3915 reflector.go:147] k8s.io/client-go@v1.28.8-k3s1/tools/cache/reflector.go:229: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": read tcp 127.0.0.1:39464->127.0.0.1:6444: read: connection reset by peer - error from a previous attempt: read tcp 127.0.0.1:42520->127.0.0.1:6444: read: connection reset by peer
Apr 03 16:20:12 node1 k3s[3915]: time="2024-04-03T16:20:12Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": EOF"
Apr 03 16:20:13 node1 k3s[3915]: time="2024-04-03T16:20:13Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:20:13 node1 k3s[3915]: time="2024-04-03T16:20:13Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:13 node1 k3s[3915]: time="2024-04-03T16:20:13Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:17 node1 k3s[3915]: time="2024-04-03T16:20:17Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:39680->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:20:18 node1 k3s[3915]: time="2024-04-03T16:20:18Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:20:18 node1 k3s[3915]: time="2024-04-03T16:20:18Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:18 node1 k3s[3915]: time="2024-04-03T16:20:18Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:22 node1 k3s[3915]: time="2024-04-03T16:20:22Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:57740->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:20:23 node1 k3s[3915]: time="2024-04-03T16:20:23Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:20:23 node1 k3s[3915]: time="2024-04-03T16:20:23Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:23 node1 k3s[3915]: time="2024-04-03T16:20:23Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:27 node1 k3s[3915]: time="2024-04-03T16:20:27Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:57920->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:20:28 node1 k3s[3915]: time="2024-04-03T16:20:28Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:20:28 node1 k3s[3915]: time="2024-04-03T16:20:28Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:28 node1 k3s[3915]: time="2024-04-03T16:20:28Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:32 node1 k3s[3915]: time="2024-04-03T16:20:32Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://127.0.0.1:6444/cacerts\": read tcp 127.0.0.1:58574->127.0.0.1:6444: read: connection reset by peer"
Apr 03 16:20:33 node1 k3s[3915]: time="2024-04-03T16:20:33Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:20:33 node1 k3s[3915]: time="2024-04-03T16:20:33Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"
Apr 03 16:20:33 node1 k3s[3915]: time="2024-04-03T16:20:33Z" level=error msg="Remotedialer proxy error" error="dial tcp 10.0.2.15:6443: connect: connection refused"
@pgregorio-signalytic
Copy link
Author

I believe there were issues with older iptables but when I looked at my version, it seems to be okay:

root@master:/home/vagrant# iptables --version
iptables v1.8.7 (nf_tables)

@brandond
Copy link
Member

brandond commented Apr 3, 2024 •
edited
Loading 

Apr 03 16:18:57 node1 k3s[3915]: time="2024-04-03T16:18:57Z" level=info msg="Connecting to proxy" url="wss://10.0.2.15:6443/v1-k3s/connect"
Apr 03 16:18:57 node1 k3s[3915]: time="2024-04-03T16:18:57Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp 10.0.2.15:6443: connect: connection refused"

The agent can't connect to the server. Did you need to use --node-ip on the server to tell it to use the IP address of a different interface for cluster communication? Or do you need to open up ports on that node so that the agent can connect?

@k3s-io k3s-io locked and limited conversation to collaborators Apr 3, 2024
@brandond brandond converted this issue into discussion #9861 Apr 3, 2024
@github-project-automation github-project-automation bot moved this from New to Done Issue in K3s Development Apr 3, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Assignees
No one assigned
Labels
None yet
Projects
K3s Development
Archived in project
Milestone
No milestone
Development

No branches or pull requests
2 participants
@brandond @pgregorio-signalytic