refactor(tests): Improve performance when obtaining bitcoind binary

[moisesPompilio]

Description and Notes

This PR refactors ./test/prepare.sh to significantly improve performance when obtaining bitcoind. Instead of always cloning and building Bitcoin Core, the script now:

• Checks if the user provided a custom bitcoind path.
• If not, attempts to download a supported prebuilt binary for the current OS and version.
• Falls back to cloning and building Bitcoin Core only when needed.

This reduces build time during local development and CI. Functional tests currently take about 12 minutes, and with these changes the execution time should drop to roughly 8 minutes.

The script also now uses a deterministic binaries directory instead of generating a new path per commit. Since only Floresta changes on each run and prepare.sh always rebuilds it, a static path under /tmp/floresta-func-tests makes the process faster and more consistent.

Another change is that the default build mode for Floresta is now debug. Release builds should be explicitly triggered with the --release flag. This avoids expensive build times when iterating on tests locally while still allowing CI and production-level testing to use optimized builds. All updates are documented.

How to verify the changes you have done?

1. Use a custom bitcoind:

# The script should skip downloading or building Bitcoin Core and use the provided binary.
BITCOIND_EXE=/path/to/bitcoind ./tests/prepare.sh

2. Download a prebuilt binary:

# A prebuilt binary should be downloaded if the version and platform are supported.
BITCOIN_REVISION=28.3 ./tests/prepare.sh

3. Build Bitcoin Core from source:

# If no prebuilt binary exists, the script should clone and build Bitcoin Core automatically.
BITCOIN_REVISION=28.0 BUILD_BITCOIND_NPROCS=2 ./tests/prepare.sh

4. Build Floresta in release mode:

./tests/prepare.sh --release

[jaoleal]

Concept ACK.

Just two things

1. this increses the complexity for this shell script in a point where i start questioning if we should take a step back and aim another way to make these scripts. IMO shell is fine for me as long we keep its dependencies organized.
2. The paths were already deterministic, thats why it used git hashes instead of random numbers, its necessary to avoid cleaning for every run, versioning conflicts and to seek truly deterministic results on our test (this last part is a far dream, our test framework have 30% chance of not failing lol). You can improve usage and readability by keeping the static directory but using short git commits as subdirectories for storing logs and binaries.

[qlrd]

cACK

Just a minor thing about the path of binaries

[moisesPompilio]

1. this increses the complexity for this shell script in a point where i start questioning if we should take a step back and aim another way to make these scripts. IMO shell is fine for me as long we keep its dependencies organized.

I agree the script is getting more complex, but the real issue is how we obtain the bitcoind binary. Building Bitcoin manually is heavy, requires git cloning and compiling, and wastes time for both developers and CI. I think we should keep only two options: letting the user provide a path to their own bitcoind binary, or downloading a known binary we support. Compiling inside the script brings no real benefit, adds error-handling complexity, and creates compatibility issues with older versions that don’t support CMake. With these two options we cover all use cases without increasing script complexity.

2. The paths were already deterministic, thats why it used git hashes instead of random numbers, its necessary to avoid cleaning for every run, versioning conflicts and to seek truly deterministic results on our test (this last part is a far dream, our test framework have 30% chance of not failing lol). You can improve usage and readability by keeping the static directory but using short git commits as subdirectories for storing logs and binaries.

I see these commit-tagged directories more as a workaround than something that actually solves the real problems. The issues come from the tests or the code itself. One reason this appears to work is that every branch change or commit creates a new log path, so test functions that read logs avoid breaking because the logs stay small. But this is already being fixed in PR 671, which reduces log size and prevents those failures. Outside of that, using commit tags for directories doesn’t add real value.

[jaoleal]

I see these commit-tagged directories more as a workaround than something that actually solves the real problems. The issues come from the tests or the code itself. One reason this appears to work is that every branch change or commit creates a new log path, so test functions that read logs avoid breaking because the logs stay small. But this is already being fixed in PR #671, which reduces log size and prevents those failures. Outside of that, using commit tags for directories doesn’t add real value.

The decision to use git tags that i made when i wrote this prepare.sh was to produce the same result every time that i run the command to execute the tests so in the future we could do some interesting things such as publishing some hashes of these directories and one using floresta could attest it. (the main idea that i had)

We have a directory to store the results of the tests and we need to have some kind of organization that help us to avoid conflicts and to accompany the changes. Random numbers do accomplish this but they arent meaningfull at all and you dont have any way to predict them (because they are random) and thats why i used git tags, they have more utility, please keep them.

The real upgrades being introduced here doesnt conflict with git tags at all.

[moisesPompilio]

The decision to use git tags that i made when i wrote this prepare.sh was to produce the same result every time that i run the command to execute the tests so in the future we could do some interesting things such as publishing some hashes of these directories and one using floresta could attest it. (the main idea that i had)

This isn’t really feasible. Tests that mine blocks break determinism because the block timestamps always differ, so the reproduced results won’t match. And even conceptually, hashing these outputs doesn’t add much value. The purpose of the tests is to validate behavior, and they should pass consistently everywhere.

We have a directory to store the results of the tests and we need to have some kind of organization that help us to avoid conflicts and to accompany the changes. Random numbers do accomplish this but they arent meaningfull at all and you dont have any way to predict them (because they are random) and thats why i used git tags, they have more utility, please keep them.

The real upgrades being introduced here doesnt conflict with git tags at all.

The upgrade indeed improves performance for CI and for local runs. Keeping the test directories static improves it even further for local use, since you no longer need to rebuild Utreexod or re-download bitcoind every time you switch branches or make a commit. The only thing that needs rebuilding is Floresta itself, which is exactly what should happen. This gives a significant boost for local workflows.

[jaoleal]

This isn’t really feasible. Tests that mine blocks break determinism because the block timestamps always differ, so the reproduced results won’t match. And even conceptually, hashing these outputs doesn’t add much value. The purpose of the tests is to validate behavior, and they should pass consistently everywhere.

I think you already have noted that our tests arent ideal and expecting them to work in an ideal setup is logically broken. If the concern is to be ideal, performatic etc etc why are you even considering a good approach to literally mine a block during tests ?

Theres a lot of work to do in order to not have any contradictions in how we are doing tests but right now is doing the job that its needing to be done... Introducing too much changes expecting for it to fit an ideal state will only create a development burden for something thats not the focus point of the project, the project is not even released... We need to do an incremental work without breaking too many things and unecessary changes that doesnt really bring any usefull upgrade at all.

[Davidson-Souza]

The decision to use git tags that i made when i wrote this prepare.sh was to produce the same result every time that i run the command to execute the tests so in the future we could do some interesting things such as publishing some hashes of these directories and one using floresta could attest it. (the main idea that i had)

They will never be that deterministic, and I don't think that's necessary. We can reproduce errors without needing perfect determinism. The reason why I liked the tags was so you could keep multiple runs saved on-disk and revisit them as needed. I actually keep logs for different runs when I'm developing something new.

Perhaps we can leave the root folder and bins folder without the tag, and having subdirs for each run, prefixed by the commit.

[jaoleal]

They will never be that deterministic, and I don't think that's necessary.

Its not that necessary but IMHO we should keep it in mind, I dont think is any good to have tests that differ in results... Certainly theres a way to accomplish a certain level of certainty of its results.

Perhaps we can leave the root folder and bins folder without the tag, and having subdirs for each run, prefixed by the commit.

Yeah, proposed that in #716 (comment)

[moisesPompilio]

@jaoleal @Davidson-Souza I set up the paths for the logs based on git describe, so each run will have a log specific to its commit. Everything still goes into /tmp/floresta-func-tests/ to keep it organized. The log folder management is handled by the Python code. I could do the same for the /data directory if needed; for example, if we ever want to save node states. But I'm not sure that's necessary right now, since the /data directory is deleted at the start of each test run.

[jaoleal]

I ran the script some times locally, its pretty good!

[jaoleal]

I could do the same for the /data directory if needed; for example, if we ever want to save node states. But I'm not sure that's necessary right now, since the /data directory is deleted at the start of each test run.

Its cool to have the possibility to share states between tests cc @j-moreno-c-r

[moisesPompilio]

Its cool to have the possibility to share states between tests cc @j-moreno-c-r

Removing the /data directory is essential to ensure that each test run starts all nodes fresh and without persistence. This prevents issues like nodes already having blocks and causing tests to fail because of leftover state. However, tests can still share a node between them if they're configured to do so.

[moisesPompilio]

I removed downloading the essential Bitcoin Core components. Now it only installs libboost-all-dev, which is required when we start using bitcoinkernel, as done in PR #713

[Davidson-Souza]

Needs rebase

[moisesPompilio]

I did the rebase.

[moisesPompilio]

Version 30.0 is no longer being distributed due to a CVE found in it, so the latest current version is 29.2. Therefore I changed the default version from 30.0 to 29.2.

[moisesPompilio]

I made adjustments to the code. Instead of checking hashes that are already in prepare.sh, now we have files with all the hashes for that version that we can use. They're located in ./tests/bitcoin_hashes and are separated by bitcoind version. Additionally, now a branch can be passed in BITCOIN_REVISION to be compiled in bitcoin, so it's possible to do compatibility tests with the master branch of bitcoin core, for example.

[Davidson-Souza]

Version 30.0 is no longer being distributed due to a CVE found in it, so the latest current version is 29.2. Therefore I changed the default version from 30.0 to 29.2.

They now have a 30.2 available: https://bitcoincore.org/en/2026/01/10/release-30.2/

[Davidson-Souza]

In 3a4c028 commit message you say: Implemented two modes for obtaining the bitcoind binary:. But I see three

[Davidson-Souza]

Also: for versions > 30, can we disable IPC for now?

[Davidson-Souza]

I ran with the build from source option, and even though the Bitcoin Core build failed, it kept going. I think it should abort in this case

[moisesPompilio]

I updated the default bitcoin version to 30.2 which is the most recent, modified the script so that in case of an error in the bitcoin core compilation it stops the execution, and disabled IPC for versions higher than 29.

[jaoleal]

ACK 0ec125c

[Davidson-Souza]

ACK 0ec125c