Migrate functional tests from Python to Rust with integrated RPC and node setup #690
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Fix strippedsize to correctly calculate and return the block size excluding witness data (previously returned total block size) - Ensure bits, chainwork, and version_hex fields return data in big-endian hex order for proper serialization - Adjust difficulty to use the appropriate calculation method for accurate float representation - Add target field to include the block's target value in big-endian hex, which was missing before
Make the fields in GetMemInfoStats and MemInfoLocked public to allow implementations using floresta_rpc to access getmemoryinfo in the rpc
Make the fields in ActiveCommand and GetRpcInfoRes public to allow implementations to access RPC information details via the getrpcinfo method.
…tegration - Created a new workspace member `test-rust` for functional testing. - Added `Cargo.toml` for `test-rust` with dependencies for testing. - Implemented a basic `main.rs` file for the test project. - Developed common utilities for setting up Bitcoind and Florestad nodes. - Implemented Utreexod setup and RPC client for interaction with the Utreexod node. - Added tests for Bitcoind and Florestad integration, including peer connection checks. - Created functions to generate blocks and manage node interactions. - Established a shared setup for tests to optimize resource usage.
These tests ensure that Floresta's RPC responses align with Bitcoin Core's RPC for maintaining consensus.
Introduce Rust functional tests in the CI workflow using a matrix strategy for parallel execution with Python tests. Add scripts to prepare and compile necessary binaries for Utreexod and Bitcoind
Collaborator
Author
|
I opened this PR initially as a draft to validate the overall approach and confirm that the direction taken meets expectations. Please feel free to share any feedback or suggestions, particularly about the test structure, CI setup, or integration strategy. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of this pull request?
Which crates are being modified?
Description and Notes
This PR migrates the existing Python functional tests to Rust to align with the project’s ecosystem and take advantage of Rust’s testing capabilities. Since Floresta and its supporting components are written in Rust, keeping tests in Python added unnecessary complexity and dependency management.
The new Rust-based functional test suite can automatically instantiate a bitcoind, utreexod, and florestad node, simulating full regtest integration. It validates the interoperability between these nodes and verifies that Floresta’s RPC responses match those of Bitcoin Core, essential since Floresta aims to maintain RPC compatibility with Core.
A shared static setup,
SHARED_FLORESTAD_BITCOIND_UTREEXOD_WITH_BLOCKS, is provided for tests that require a pre-initialized chain with blocks, improving runtime performance by avoiding repeated setup overhead.The following RPC endpoints are covered:
add_nodeget_best_block_hashget_blockget_blockchain_infoget_block_countget_block_hashget_block_headerget_memory_infoget_peer_infoget_rootsget_rpc_infoget_tx_outpingstopuptimeThe tests use the electrsd crate to manage the bitcoind node. This library (also used by BDK and LDK) automatically downloads and launches a bitcoind binary matching the desired version, simplifying setup and communication via RPC.
In CI, a preparation step was added to download the Bitcoin binaries and build utreexod, placing them under /tmp/floresta-func-tests/bin. The CI uses prebuilt binaries for Linux and macOS to reduce build time (Linux only for now).
Additionally, the minimum Rust version was updated to 1.90.0, as some dependencies required a newer compiler. This update aligns the project with the latest stable toolchain.
Tests are isolated behind the functional-tests feature to prevent them from running as part of the standard test suite, since they require external binaries and setup.
This PR depends on #689 #688 #683 #682
How to verify the changes you have done?
Run the functional test suite with the following:
cargo test --features=functional-tests -p test-functional -- --test-threads=1Alternatively, use prepare.sh to download and prepare the required binaries before running. The tests will automatically start bitcoind, utreexod, and florestad instances and validate RPC responses against bitcoind.