modernize eventrouter

[vigith]

We internally rely on eventrouter to log events to stdout, which is then relayed to our central logging platform. Then came a requirement that we have to publish the events to couple of other sinks for more processing (streaming and big data). As we looked into the source code to contribute/extend we felt it would be great to make following core changes too.

Please let me know whether the following changes resonate well with you.

Remove Vendoring

• Today upgrading between go version brings in huge import PRs. Lots of vendor files are changed.
• Adding new sinks also adds new modules in vendor directory.

If we move to go 1.13 which supports module mirroring, we can remove vendoring all together (quite close to #80).

Multiple Sink Support

There are few use cases where we would like to send the same event to multiple sinks simultaneously, eg. send the events to local file, Kafka and say, AWS Kinesis. In the current implementation, we will have to run multiple eventrouters to send to multiple sinks simultaneously. (#79)

This will be bring a problem on how to deal, where we are able to write to 1 sink while the other fails. Simplest solution is to ask each individual sink writers to retry if they see a failure. Also the writes should be async so that a slow sink won't slow down the rest.

Controller structure

The official Kubernetes GitHub repository has a nice controller sample-controller design diagram, which employs following components.

Workqueue

Workqueue will help us implement queuing and retrying mechanism.

Cache

Repeatedly fetching information from the API server can become expensive and can be alleviated using cache.

ListWatch

eventrouter use informers.NewSharedInformerFactory which listens to all the Namespaces

	sharedInformers := informers.NewSharedInformerFactory(clientset, resync)
	eventsInformer := sharedInformers.Core().V1().Events()

We could move to cache.ListWatch and provide ListFunc and WatchFunc to filter out namespaces.

Improved Testing

Currently there is no code coverage for controller in eventrouter, adding more tests will be very helpful to boost the confidence during refactoring.

[vigith]

Hi @timothysc, How would you like me to proceed with this one? Open 1 PR at a time? I have opened 1 already.

[genebean]

@vigith what did you all end up doing? Are you still using this or running a fork or running some other tool?

[vigith]

We ended up using our own customized version.

[genebean]

Is your version public?

[vigith]

unfortunately no because we had to encrypt the payload due to our security policies before posting to Kafka.