Add UDP source IP into golang dialer https connection for Edgeview

[naiming-zededa]

• have seen on certain EVE devices, even walk through various ports, the UDP source IP address used stuck on one particular IP address, and it then failed on DNS query and failed Edgeview connection.
• change the retry only after walk through all the intfs to speed up the initial connection

[eriknordmark]

Given that we have this functionality (plus the functionality to do DNS lookups per interface, and caching) in pkg/pillar/zedcloud/send.go, would it make sense to use that package?

[naiming-zededa]

Given that we have this functionality (plus the functionality to do DNS lookups per interface, and caching) in pkg/pillar/zedcloud/send.go, would it make sense to use that package?

I think this code is only for websocket connection for Edgeview, not to send the packets, once connected then it's done. It is not easy to change to the send.go's functions. Also this edgeview does not subscribe pillar services, like devicenetworkstatus in the current implementation. The original intention is to run this even on generic linux machines.

[eriknordmark]

I think this code is only for websocket connection for Edgeview, not to send the packets, once connected then it's done. It is not easy to change to the send.go's functions. Also this edgeview does not subscribe pillar services, like devicenetworkstatus in the current implementation. The original intention is to run this even on generic linux machines.

Then I don't understand. Setting the source address is useful for selecting a network port in EVE-OS (because we configure PBR rules for this purpose), but it doesn't do that on a generic linux machine - it merely sets the source address but the traffic might be routed out the wrong port.

[naiming-zededa]

I think this code is only for websocket connection for Edgeview, not to send the packets, once connected then it's done. It is not easy to change to the send.go's functions. Also this edgeview does not subscribe pillar services, like devicenetworkstatus in the current implementation. The original intention is to run this even on generic linux machines.

Then I don't understand. Setting the source address is useful for selecting a network port in EVE-OS (because we configure PBR rules for this purpose), but it doesn't do that on a generic linux machine - it merely sets the source address but the traffic might be routed out the wrong port.

The logic here in Edgeview is get the default route pointed interfaces, and get their port ip addresses of those interfaces. then walk through them one by one, plus an extra one does not use any source ip address. Say the device has default route points out through 'eth1' and 'eth2', so this edgeview connection walk, will be using the source ip of eth1, then eth2, then no source ip. It will keep try those three options. A simple way compare to current EVE send packets to controller.

[eriknordmark]

The logic here in Edgeview is get the default route pointed interfaces, and get their port ip addresses of those interfaces. then walk through them one by one, plus an extra one does not use any source ip address. Say the device has default route points out through 'eth1' and 'eth2', so this edgeview connection walk, will be using the source ip of eth1, then eth2, then no source ip. It will keep try those three options. A simple way compare to current EVE send packets to controller.

But that is useless on Linux out of the box.
The fact that the source IP is set to the IP address on eth0 has no effect on routing. Thus your three attempts might all be routed out eth1 - there isn't any round-robin in the kernel when selecting among multiple default routes.

This works on EVE-OS because we set up ip rules to use a different routing table based on the source IP address. That is done so that the routing table used only has the routes which point out that particular port. So your code is useful on EVE-OS but not on Linux itself.

Also, on EVE-OS, in addition to the interface selection the code in send.go also makes the DNS lookups use the matching interface and we presumably need that logic for EdgeView as well for it to be robust in e.g., LTE/5G setups with walled garden DNS servers. That was why I was wondering if you could use that package at least when building for EVE-OS.

[naiming-zededa]

Yes, this source IP scheme will only work in EVE-OS currently. But normal linux probably does not have this complexity, and the intention was not to include lots of specific import from EVE. Although looking at the send.go implementation, they are mostly internal to usage of when sending packets to, it is not easy to convert this facility into the Edgeview code in the current format. I can make a comment on this.

[eriknordmark]

Yes, this source IP scheme will only work in EVE-OS currently. But normal linux probably does not have this complexity, and the intention was not to include lots of specific import from EVE. Although looking at the send.go implementation, they are mostly internal to usage of when sending packets to, it is not easy to convert this facility into the Edgeview code in the current format. I can make a comment on this.

OK.

[eriknordmark]

LGTM

[naiming-zededa]

@eriknordmark should this be ported into 12 and/or 13 stable? since we have seen this issue on customer's devices.