cmake: Regular rebasing of the cmake-staging branch
#280
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We need to synchronize between various tx download structures. TxRequest does not inherently need cs_main for synchronization, and it's not appropriate to lock all of the tx download logic under cs_main.
This is a synchronous callback notifying clients of all tip changes. It allows clients to respond to a new block immediately after it is connected. The synchronicity is important for things like m_recent_rejects, in which a transaction's validity can change (rejected vs accepted) when this event is processed. For example, the transaction might have a timelock condition that has just been met. This is distinct from something like m_recent_confirmed_transactions, in which the validation outcome is the same (valid vs already-have), so it does not need to be reset immediately.
Resetting m_recent_rejects once per block is more efficient than comparing hashRecentRejectsChainTip with the chain tip every time we call AlreadyHaveTx. We keep hashRecentRejectsChainTip for now to assert that updates happen correctly; it is removed in the next commit.
This also means AlreadyHaveTx no longer needs cs_main held.
The TxOrphanage is now guarded externally by m_tx_download_mutex.
If the binaries don't exist, the Guix build has failed for some other reason. There's no need to check for unknown architectures, or executable formats, as the only ones that could be built are those that we've configured toolchains for in Guix. We've also been doing this inconsistently across the two scripts.
…jection filters once per tip change c85acce [refactor] delete EraseTxNoLock, just use EraseTx (glozow) 6ff8406 remove obsoleted TxOrphanage::m_mutex (glozow) 61745c7 lock m_recent_confirmed_transactions using m_tx_download_mutex (glozow) 723ea0f remove obsoleted hashRecentRejectsChainTip (glozow) 18a4355 update recent_rejects filters on ActiveTipChange (glozow) 36f170d add ValidationInterface::ActiveTipChange (glozow) 3eb1307 guard TxRequest and rejection caches with new mutex (glozow) Pull request description: See bitcoin#27463 for full project tracking. This contains the first few commits of bitcoin#30110, which require some thinking about thread safety in review. - Introduce a new `m_tx_download_mutex` which guards the transaction download data structures including `m_txrequest`, the rolling bloom filters, and `m_orphanage`. Later this should become the mutex guarding `TxDownloadManager`. - `m_txrequest` doesn't need to be guarded using `cs_main` anymore - `m_recent_confirmed_transactions` doesn't need its own lock anymore - `m_orphanage` doesn't need its own lock anymore - Adds a new `ValidationInterface` event, `ActiveTipChanged`, which is a synchronous callback whenever the tip of the active chainstate changes. - Flush `m_recent_rejects` and `m_recent_rejects_reconsiderable` on `ActiveTipChanged` just once instead of checking the tip every time `AlreadyHaveTx` is called. This should speed up calls to that function (no longer comparing a block hash each time) and removes the need to lock `cs_main` every time it is called. Motivation: - These data structures need synchronization. While we are holding `m_tx_download_mutex`, these should hold: - a tx hash in `m_txrequest` is not also in `m_orphanage` - a tx hash in `m_txrequest` is not also in `m_recent_rejects` or `m_recent_confirmed_transactions` - In the future, orphan resolution tracking should also be synchronized. If a tx has an entry in the orphan resolution tracker, it is also in `m_orphanage`, and not in `m_txrequest`, etc. - Currently, `cs_main` is used to e.g. sync accesses to `m_txrequest`. We should not broaden the scope of things it locks. - Currently, we need to know the current chainstate every time we call `AlreadyHaveTx` so we can decide whether we should update it. Every call compares the current tip hash with `hashRecentRejectsChainTip`. It is more efficient to have a validation interface callback that updates the rejection filters whenever the chain tip changes. ACKs for top commit: instagibbs: reACK c85acce dergoegge: Code review ACK c85acce theStack: Light code-review ACK c85acce hebasto: ACK c85acce, I have reviewed the code and it looks OK. Tree-SHA512: c3bd524b5de1cafc9a10770dadb484cc479d6d4c687d80dd0f176d339fd95f73b85cb44cb3b6b464d38a52e20feda00aa2a1da5a73339e31831687e4bd0aa0c5
1bc9f64 contrib: assume binary existence in sec/sym checks (fanquake) 51d8f43 contrib: simplify ELF test-security-check (fanquake) 1810e20 contrib: simplify PE test-security-check (fanquake) 6c9746f contrib: simplify MACHO test-security-check (fanquake) Pull request description: The current `test-security-check` script is hard to understand, and change (i.e https://github.com/bitcoin/bitcoin/pull/29987/files#diff-52aa0cda44721f089e53b128cb1232a876006ef257b211655456b17dfb2ec712); tests are also not done in isolation (when-possible). Fix that, and add missing checks. Simplifies future toolchain/security/hardening changes. ACKs for top commit: hebasto: ACK 1bc9f64 (assuming my Guix hashes match; I'll provide them shortly). TheCharlatan: ACK 1bc9f64 Tree-SHA512: 1885d0ce63a94ffa61345327f919da20b63de6dd4148d6db3ee8bad4485253a36e8ab0dbee48cecc02ea35d139edfed75453af45fc364bcbef6fe16b6823bc7a
Also add a sanity check for non-encapsulated (directory-wide) build properties.
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com> Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
This reverts commit 1155978.
Our `QT_STATICPLUGIN` macro is effectively equivalent to the Qt's `QT_STATIC` macro. No need to handle both of them.
When using CMake, each plugin comes with a C++ stub file that automatically initializes the static plugin. Consequently, any target that links against a plugin has this C++ file added to its SOURCES, which makes the removed code redundant.
To configure CMake for cross-compiling, use
`--toolchain depends/${HOST}/toolchain.cmake` command-line option.
The content of those variables is appended to the each target after the flags added by the build system.
Keep this commit at the top when rebasing.
This change fixes CI jobs as the `miniupnpc` source archive is not available neither at https://miniupnp.tuxfamily.org nor at https://bitcoincore.org/depends-sources at this moment.
|
Friendly ping @TheCharlatan @m3dwards :) |
|
ACK dc490da I did get a small diff in Will update with Guix build results when they are baked. |
|
Guix build 5cef021ba00f903d7633096663febd72853c4584499b8f25ae42ae6dc36f82cf guix-build-dc490dae00d6/output/aarch64-linux-gnu/SHA256SUMS.part
9cdb2e684b2e0a70c25ec632c1ba346550ee620da8018022e69d455536a4615c guix-build-dc490dae00d6/output/aarch64-linux-gnu/bitcoin-dc490dae00d6-aarch64-linux-gnu-debug.tar.gz
4da2aed25f0348fb4ebe1b2af01b2a5ce45ddb696c4654d83206e526eb59f02f guix-build-dc490dae00d6/output/aarch64-linux-gnu/bitcoin-dc490dae00d6-aarch64-linux-gnu.tar.gz
9f5da3540c02dce6f46908a09b91a1e83ddf65e96e21b900f9eaf3d5c63925b6 guix-build-dc490dae00d6/output/arm-linux-gnueabihf/SHA256SUMS.part
17c4ea025c1fade50345056ae1bdb9f84f7a0f229b10131844131ced253a3101 guix-build-dc490dae00d6/output/arm-linux-gnueabihf/bitcoin-dc490dae00d6-arm-linux-gnueabihf-debug.tar.gz
b2336f5405d5a7b8628f0e9d1b37a4f5e113de06e900fa6cd31f38a979aeeb3d guix-build-dc490dae00d6/output/arm-linux-gnueabihf/bitcoin-dc490dae00d6-arm-linux-gnueabihf.tar.gz
51e4ef6ea6395c69aeb68a3645ce106b56b54d3185be0dbc7fb1802cb8c54b57 guix-build-dc490dae00d6/output/arm64-apple-darwin/SHA256SUMS.part
3677aada031d03766b76ccc57731ecf8c6de77323a65027d7e1feff61f832200 guix-build-dc490dae00d6/output/arm64-apple-darwin/bitcoin-dc490dae00d6-arm64-apple-darwin-unsigned.tar.gz
433ea9ef28d33469f8c6e246952515ebe77f02ff9873aa140091063ff30e7aa7 guix-build-dc490dae00d6/output/arm64-apple-darwin/bitcoin-dc490dae00d6-arm64-apple-darwin-unsigned.zip
c3fb9574d14b24ac6c4949d1d30ecdaaccba128cb18a2c963ff6e9d425e61ea1 guix-build-dc490dae00d6/output/arm64-apple-darwin/bitcoin-dc490dae00d6-arm64-apple-darwin.tar.gz
c6057ab3970124cd001c5cd5c0d9a73e7651fcd27e79a47c16aef9f3ad14262c guix-build-dc490dae00d6/output/dist-archive/bitcoin-dc490dae00d6.tar.gz
f2788e6c4403fa1f8696ba89dccc2f1aa0fe1568a3545585d323bb21b76711ef guix-build-dc490dae00d6/output/powerpc64-linux-gnu/SHA256SUMS.part
479db9502720ed8c4a3206563525061071bff29f4dbdc802978af94e751be1d7 guix-build-dc490dae00d6/output/powerpc64-linux-gnu/bitcoin-dc490dae00d6-powerpc64-linux-gnu-debug.tar.gz
f939b9a5525ea860b35e238b6599bce1ee6a871ae43cc991f1ffe8fa3bdbf671 guix-build-dc490dae00d6/output/powerpc64-linux-gnu/bitcoin-dc490dae00d6-powerpc64-linux-gnu.tar.gz
8806e3ad997f3fedd42415fff8899eb3ad19eff24848938197345e0c6d695d58 guix-build-dc490dae00d6/output/riscv64-linux-gnu/SHA256SUMS.part
67beb8c24aea5356f93a60dde85d7f5b53ba3899dd2929a29d0af53e9cccc026 guix-build-dc490dae00d6/output/riscv64-linux-gnu/bitcoin-dc490dae00d6-riscv64-linux-gnu-debug.tar.gz
64bab910ce4726da378a29f131d2aef0da5e8867172cfc9b3aa6ce4f8a1f2b3c guix-build-dc490dae00d6/output/riscv64-linux-gnu/bitcoin-dc490dae00d6-riscv64-linux-gnu.tar.gz
e0b23586b5345097f69938b9f4b647dd0cf4dad57d1268e3f97e547e778547ef guix-build-dc490dae00d6/output/x86_64-apple-darwin/SHA256SUMS.part
7efc5d886e43693a4fe33492741ab4034fd4c6ac529508eb3412679b423df3e5 guix-build-dc490dae00d6/output/x86_64-apple-darwin/bitcoin-dc490dae00d6-x86_64-apple-darwin-unsigned.tar.gz
9350523c6b4680c91903281c7bb16b26153a8b4f7dc5bda56c59f8d52a01cc0a guix-build-dc490dae00d6/output/x86_64-apple-darwin/bitcoin-dc490dae00d6-x86_64-apple-darwin-unsigned.zip
3bc7106aceea901d942041c8561fe2fb79072a9ee2c324344d1b7cc36905bda8 guix-build-dc490dae00d6/output/x86_64-apple-darwin/bitcoin-dc490dae00d6-x86_64-apple-darwin.tar.gz
21aa524795a3ae519914729692793b8221f632cffa87670a0a6c698311177d75 guix-build-dc490dae00d6/output/x86_64-linux-gnu/SHA256SUMS.part
f6b46973ecd4d288fc4f5830402a4769de96a08b6729fe437054398bc363d7a4 guix-build-dc490dae00d6/output/x86_64-linux-gnu/bitcoin-dc490dae00d6-x86_64-linux-gnu-debug.tar.gz
9be915b3f65fb902b49d0ae5b2e8a910f883eb3a61a537e2e9fe6dd27e19f8f7 guix-build-dc490dae00d6/output/x86_64-linux-gnu/bitcoin-dc490dae00d6-x86_64-linux-gnu.tar.gz
21f26e55a281e4334771106369d7190406c78d25d0df5f54a6523a41f539d0f6 guix-build-dc490dae00d6/output/x86_64-w64-mingw32/SHA256SUMS.part
a7e8fb83597e22cf503e273117a9b242676da312bfde5832ef926e6b64261a0a guix-build-dc490dae00d6/output/x86_64-w64-mingw32/bitcoin-dc490dae00d6-win64-debug.zip
f5ec21360dd9d1dc255f9cadf89a9d48659268256ad3f53fa319b3ecb669c22b guix-build-dc490dae00d6/output/x86_64-w64-mingw32/bitcoin-dc490dae00d6-win64-setup-unsigned.exe
fa4aa5de4365887cfc268277353e2c63d219cd3b7e2a24dc26aa7734e692881a guix-build-dc490dae00d6/output/x86_64-w64-mingw32/bitcoin-dc490dae00d6-win64-unsigned.tar.gz
90e7f8fd428cdbdd04c6114744abfbfd3b648d6bf19f40d5eade59e270357429 guix-build-dc490dae00d6/output/x86_64-w64-mingw32/bitcoin-dc490dae00d6-win64.zip |
|
My Guix build: |
|
Thank you for your review! This branch has been force pushed into the https://github.com/hebasto/bitcoin/tree/cmake-staging. Closing. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the cmake-staging branch rebased on the recent bitcoin/master one, which includes the main build system changes, with the squashed "!fixup" commits.
Steps to reproduce:
There are a few conflicts that have to be resolved.