Speedup tests: execute OS independent code once #486
Conversation
There was a problem hiding this comment.
I have mixed feelings about this. On the one hand, we're certainly testing way too many combinations that really don't need to be tested. On the other hand, it can also be very easy to miss it when we do become more fact dependent.
Another option is to introduce an alternative to on_supported_os. Something like on_reduced_supported_os that could be smarter and filter out some combinations: if RHEL is supported and also all compatible versions then it's sufficient to only test RHEL. That's something we can also reuse in many other modules to cut down on CI testing.
spec/defines/network_spec.rb
Outdated
There was a problem hiding this comment.
At least this file isn't entirely fact independent since it uses systemd_internal_services.
There was a problem hiding this comment.
You're right, but $fact['systemd_internal_services'] isn't defined in any OS family hiera data/. During tests it's currently using the same value from default_module_facts.yml for all test cases.
| selinux_ignore_defaults: false | ||
| ) | ||
| } | ||
| _, facts = on_supported_os.first |
There was a problem hiding this comment.
Perhaps you can introduce a helper method default_os_facts. I also wouldn't mind knowing which OS is being tested on.
When you add new feature, you should write tests to cover that feature. By executing the same tests many times with different OS facts, you won't improve quality of tests (when the code doesn't depend on those facts).
Having a method like |
This PR might improve speed of rspec tests, ~2k test cases removed.
OS independent tests will be executed only once instead of running for each supported platform.