testenv - testing environment manager

[raid-7]

See readme
What it enables

[SamuelMarks]

Are the CI servers we use powerful enough to run docker inside them? - Will this significantly slow-down the CI process?

Otherwise we might as well stick to your scripts (which I modified and extracted into its own repo: https://github.com/offscale/kv-travis-scripts)

Also I'm preparing to build dozens of little cross-platform ANSI C tools, each its own package manager, with a similar API design and idea to rvm/nvm. So then it'll just be:

etcd_vm serve latest &
consul_vm serve latest &
zookeeper_vm serve latest &

(on Windows replace & at end with start /B at beginning)

[shdown]

1. It would make more sense to be able to inject a network fault exactly at certain operation; testing what happens if the network fails at “some random time” is, in my opinion, not very useful.

2. Why all this complexity? Why do you need docker at all? Seems like ip netns (more generally, network namespaces) and ip link could do what you want, but I didn’t checked.

[raid-7]

Travis does support Docker. If we prebuild the base image and push it to the Docker registry, using Docker will likely speed up the CI process because we won't have to build vcpkg dependencies in Travis each time. However, it's the Linux-only way. To test the library on Mac OS and probably Windows in future we have to stick to the native environments.

[raid-7]

1. I suggest controlling network faults from tests. See example.

2. Yes, a network namespace with a bridge interface is the possible way too. Though, I am not sure if it would be less complex solution as long as you want to minimize manual control over running services and their addresses.

[shdown]

I suggest controlling network faults from tests. See example.

Yes, you can make the network fail between calls to library functions. What if such a function does multiple network “operations”/“requests”/whatever? It gets more complicated if you have a constantly-running stream of events, ZooKeeper-style.

[SamuelMarks]

@raid-7 I like the idea of caching the vcpkg process somehow. That seems to be the big time sync. Maybe one of you can setup a vcpkg fork that always pull down new versions, then interjects with its own CD process, create new cached tarballs which will be used by Travis instead of it's own cache for vcpkg?

Network namespacing is fine and all, but considering how basic this problem is, our current solution should suffice?

[raid-7]

What if such a function does multiple network “operations”

In this case we can design a framework with special instrumental markers and put the markers in the library functions between those operations. Something like

// perform first network request
_test_interceptor();
// perform second network request

Control the contents of the marker with macros:

#ifdef DEBUG
static void _test_interceptor() {
	TestInterceptions::call_intercepted();
}
#else
static void _test_interceptor() {}
#endif

And use them in tests like:

TEST_F(ClientFixture, conn_failure_test) {
    auto suite = TestInterceptions::create_suite();
    suite.intercept([](int call_id) {
    	ConnectionControl::instance().disconnect();
    });

    ASSERT_THROW(client->set("/key", "value"), liboffkv::ConnectionLoss);

    // suite is destroyed, interceptors are detached
}