worker auth: load correct records and ability to remove unused

[irenarindos]

Note: this will fail until the corresponding Nodeenrollment change is released

[johanbrandhorst]

This should fix the subsequent node rotation attempt, but will it fix the dialing errors we were seeing too? This relies on us having at least two active controller connections when the split brain error happens, since one will be lost during the EOF error, and the other will be required to try to rotate again. In a perfect world, I think that means the second rotation should go through and the split brain scenario should be resolved, but what if there's only a single controller and this happens? Since the worker can't tell the controller to delete unused credentials until it's able to complete dialing, I think we'll need a fix on the controller side of the dialing too.

[irenarindos]

This should fix the subsequent node rotation attempt, but will it fix the dialing errors we were seeing too? This relies on us having at least two active controller connections when the split brain error happens, since one will be lost during the EOF error, and the other will be required to try to rotate again. In a perfect world, I think that means the second rotation should go through and the split brain scenario should be resolved, but what if there's only a single controller and this happens? Since the worker can't tell the controller to delete unused credentials until it's able to complete dialing, I think we'll need a fix on the controller side of the dialing too.

It will fix the dialing errors; those were a result of GenerateServerCertificates calling LoadNodeInformation for a key id and the old logic there. We incorrectly assumed that the key id passed in was current and assigned the current auth record creds to the returned node information.