Feature request: a way to execute setup code in the test process before it becomes multithreaded

[tbottomparallel]

Motivation

Certain things cannot be done after a process becomes multithreaded. Currently there is no way that I am aware of to perform any of these tasks in a unit test because the process has multiple threads by the time any test code executes. For example:

1. std::env::set_var is set to become unsafe in the 2024 edition of Rust. The safety contract effectively requires that it only be called in a single threaded program. By the time test code is executed the process is already multithreaded. This prevents unit test code from safely setting environment variables for tests. I would argue that unit tests are one of the few places where it is reasonable for a program to modify its own environment.
   • E.g. a unit test could modify the PATH variable to point to a test directory with scripts for the purpose of validating the behavior of code which uses std::process::Command. I have used this technique in the past, the script opens a socket to communicate with the test, allowing the test to access the inputs provided to the process as well as control its output and exit code in real time. Sometimes it is not practical or even possible to avoid starting external processes, for example daemons which start and monitor other daemons necessarily must do so. This technique allows arbitrarily complex behavior to be simulated, and also prevents any of the actual binaries from accidentally being invoked.
2. CLONE_NEWUSER in Linux requires that the calling process is not threaded. By unsharing the user namespace (among others) you can simulate all sorts of privileged actions safely in unprivileged test code.

Proposal

A way to execute arbitrary per-test setup code before the test process becomes multithreaded.

Alternatives

A custom test executor could be written for these cases, but this is cumbersome and in my experience tends to discourage test coverage of these areas.

If execution of arbitrary set-up code in single-threaded mode is out of the question for unit tests, an explicit guarantee by nextest that there are no other threads concurrently writing or reading(!) the environment through functions or global variables when the test runs would allow unit tests to safely modify the environment.

Additional context

No response

[sunshowers]

Thanks. Nextest does not control what happens within in the process, and delegates to whatever the test does. Currently, the default libtest harness always creates a monitoring thread and the actual test thread (this is completely outside nextest's control).

The safety contract effectively requires that it only be called in a single threaded program. By the time test code is executed the process is already multithreaded. This prevents unit test code from safely setting environment variables for tests. I would argue that unit tests are one of the few places where it is reasonable for a program to modify its own environment.

Indeed, many of nextest's own tests modify the environment. The libtest monitoring thread doesn't access the environment while tests are running. That combined with nextest's process-per-test model means that it is fine to modify the environment in the test, as long as the test itself hasn't created any threads of its own.

CLONE_NEWUSER in Linux requires that the calling process is not threaded. By unsharing the user namespace (among others) you can simulate all sorts of privileged actions safely in unprivileged test code.

If you want one thread, or to run the test in the main thread of the process, you must use a custom test harness. See https://nexte.st/docs/configuration/extra-args/#use-case-running-tests-on-the-main-thread for more documentation.

[tbottomparallel]

I am assuming in the context of the 'double spawn' it would be possible for the second spawn to be single threaded at the time of the fork, but I may be misunderstanding how nextest works.

I'll look at libtest-mimic, sounds like that might be my best best.

[sunshowers]

I am assuming in the context of the 'double spawn' it would be possible for the second spawn to be single threaded at the time of the fork, but I may be misunderstanding how nextest works.

On most Unix platforms, there are no fork calls during process spawning -- we use posix_spawn. (fork/exec are part of Unix lore, but are a bit of an outdated way to spawn processes. The modern method is posix_spawn.)

Even if there were a fork, I believe calling setenv between fork and exec would be undefined behavior, because setenv isn't async-signal-safe. https://man7.org/linux/man-pages/man7/signal-safety.7.html

[sunshowers]

I'll look at libtest-mimic, sounds like that might be my best best.

To be clear, if you want to set environment variables at the beginning of a test, you can use the default test harness -- the safety contract is upheld in that case, even though the test process has two threads.

If you want one thread that you have to use libtest-mimic -- this is a libtest limitation, not a nextest one.

[tbottomparallel]

Yeah I see what you mean, that seems pretty clear.

At least I can confidently add a safety comment when calling env::set_var.

Thanks for the info!

[sunshowers]

I've documented this at https://nexte.st/docs/configuration/env-vars/#altering-the-environment-within-tests -- let me know if anything about it doesn't make sense.