Calico node failed to start due to no such file or directory name="/usr/local/bin/felix-plugins/felix-api-proxy"

[dylan2intel]

Environment

• Calico/VPP version: both v3.23 and v3.24
• Kubernetes version: 1.24
• Deployment type: bare-metal...
• Network configuration: Calico /VXLAN enabled

Issue description
2022-12-23 07:54:43.193 [WARNING][32] tunnel-ip-allocator/param_types.go 294: Path lookup failed error=exec: "/usr/local/bin/felix-plugins/felix-api-proxy": stat /usr/local/bin/felix-plugins/felix-api-proxy: no such file or directory name="/usr/local/bin/felix-plugins/felix-api-proxy"
2022-12-23 07:54:43.193 [ERROR][32] tunnel-ip-allocator/param_types.go 299: Executable missing
2022-12-23 07:54:43.193 [ERROR][32] tunnel-ip-allocator/config_params.go 559: Invalid (required) config value. error=Failed to parse config parameter DataplaneDriver; value "/usr/local/bin/felix-plugins/felix-api-proxy": missing file source=environment variable
2022-12-23 07:54:43.193 [PANIC][32] tunnel-ip-allocator/allocateip.go 776: Failed to parse Felix environments error=Failed to parse config parameter DataplaneDriver; value "/usr/local/bin/felix-plugins/felix-api-proxy": missing file
panic: (*logrus.Entry) 0xc00041f650

goroutine 1 [running]:
github.com/sirupsen/logrus.(*Entry).log(0xc0010ffe18, 0x0, {0xc0008fe480, 0x22})
/go/pkg/mod/github.com/sirupsen/logrus@v1.8.1/entry.go:259 +0x24f
github.com/sirupsen/logrus.(*Entry).Log(0xc00041f5e0, 0x0, {0xc0010ffe18, 0x2a423c0, 0xc000489810})
/go/pkg/mod/github.com/sirupsen/logrus@v1.8.1/entry.go:293 +0x4f
github.com/sirupsen/logrus.(*Entry).Panic(...)
/go/pkg/mod/github.com/sirupsen/logrus@v1.8.1/entry.go:331
github.com/projectcalico/calico/node/pkg/allocateip.loadFelixEnvConfig()
/go/src/github.com/projectcalico/calico/node/pkg/allocateip/allocateip.go:776 +0xa5
github.com/projectcalico/calico/node/pkg/allocateip.Run(0xc000914420)
/go/src/github.com/projectcalico/calico/node/pkg/allocateip/allocateip.go:68 +0x7b
main.main()
/go/src/github.com/projectcalico/calico/node/cmd/calico-node/main.go:169 +0x518
Calico node failed to start

To Reproduce
Steps to reproduce the behavior:

• kubectl apply -f https://projectcalico.docs.tigera.io/archive/v3.23/manifests/tigera-operator.yaml
• Download and edit file with custom cidr https://projectcalico.docs.tigera.io/archive/v3.23/manifests/custom-resources.yaml
• kubectl apply -f custom-resources.yaml

Expected behavior
Calico node successfully to start

Additional context
Add any other useful information about the problem or your deployment here.

[sknat]

Hi @dylan2intel
/usr/local/bin/felix-plugins/ should be mounted in the felix pod by the operator and the felix-api-proxy binary copied in this folder when the calico/vpp agent pod starts.

On startup the calico/vpp agent will wait for the calico/vpp vpp manager to start. What probably happens is something is preventing your VPP to start, the agent waits indefinitely, and never creates the proxy binary

Do you see anything wrong in the output of kubectl -n calico-vpp-dataplane logs calico-vpp-node-XXXX -c vpp ?

[chairwa]

problem reason might be that the following cmd was not executed:
kubectl create -f calico-vpp.yaml

[kdhanawade21]

I am also facing same issue.
Anyone got resolution for this?
Below are the logs.

2024-11-05 09:41:33.558 [INFO][23] tunnel-ip-allocator/param_types.go 289: No executable in my directory myDir="/usr/bin" name="/usr/local/bin/felix-plugins/felix-api-proxy"
2024-11-05 09:41:33.558 [INFO][23] tunnel-ip-allocator/param_types.go 297: Looking for executable on path name="/usr/local/bin/felix-plugins/felix-api-proxy"
2024-11-05 09:41:33.558 [WARNING][23] tunnel-ip-allocator/param_types.go 301: Path lookup failed error=exec: "/usr/local/bin/felix-plugins/felix-api-proxy": stat /usr/local/bin/felix-plugins/felix-api-proxy: no such file or directory name="/usr/local/bin/felix-plugins/felix-api-proxy"
2024-11-05 09:41:33.558 [ERROR][23] tunnel-ip-allocator/param_types.go 306: Executable missing
2024-11-05 09:41:33.558 [ERROR][23] tunnel-ip-allocator/config_params.go 659: Invalid (required) config value. error=Failed to parse config parameter DataplaneDriver; value "/usr/local/bin/felix-plugins/felix-api-proxy": missing file source=environment variable
2024-11-05 09:41:33.558 [PANIC][23] tunnel-ip-allocator/allocateip.go 836: Failed to parse Felix environments error=Failed to parse config parameter DataplaneDriver; value "/usr/local/bin/felix-plugins/felix-api-proxy": missing file
panic: (*logrus.Entry) 0x4000ae12d0

[kdhanawade21]

Hi @dylan2intel,
Are you able to resolve this issue?
This issue is not there on x86 architecture. I am facing this on ARM architecture.
Please let me know if you are able to resolve this issue.
Thanks in advance.