Utxo set statistics index #28
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 tasks
fjahr
force-pushed
the
utxo-stats-index
branch
from
c2469c0
to
3d3da6a
Compare
…aults on ppc64le fa40e48 ci: Remove unparseable lines from supp file for old xenial clang tsan (MarcoFalke) fa1bfc4 ci: ubsan report_error_type=1 and add suppressions (MarcoFalke) fa69cef test: Print stderr when subprocess fails (MarcoFalke) 2222c30 test: Use char instead of unsigned char (MarcoFalke) faa8023 ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le (MarcoFalke) Pull request description: Use clang-8 instead of default clang (which is clang-6 on Bionic) to avoid spurious segfaults when running the ci system on ppc64le ACKs for top commit: practicalswift: ACK fa40e48 assuming Travis is happy -- diff looks correct :) Tree-SHA512: f4f26232d3a0ef38da245869340f723d279a3db9823befbc735fb5a00024dae041c7306d7ae55d2488e6f86aa96cdea155b007aefb561fba505141e8dbc717dc
…etinfo 01c8701 util: remove unwanted fields from bitcoin-cli -getinfo (malevolent) Pull request description: Removed the following fields from -getinfo: protocolversion, walletversion and keypoololdest. This change closes bitcoin#17314 . ACKs for top commit: laanwj: ACK 01c8701 achow101: ACK 01c8701 practicalswift: ACK 01c8701 -- diff looks correct Tree-SHA512: c98f2e8a3fee04d46766f70cb88f4e49e892a4424eff8940a7d48e9e808597b702427225788f984f5c3641591fd8d86acee56774afde1d57a4259c31d971ea08
This strips some unused dylibs from bitcoin-qt. From man ld: Remove dylibs that are unreachable by the entry point or exported symbols. That is, suppresses the generation of load command commands for dylibs which supplied no symbols during the link. This option should not be used when linking against a dylib which is required at runtime for some indirect reason such as the dylib has an important initializer.
… noexcept 55b2cb1 random: mark RandAddPeriodic and SeedPeriodic as noexcept (fanquake) 461e547 doc: correct random.h docs after bitcoin#17270 (fanquake) Pull request description: The usage of `MilliSleep()` in SeedPeriodic (previously SeedSleep) was [removed](bitcoin@d61f2bb) in bitcoin#17270, meaning it, and its users can now be marked `noexcept`. This also corrects the docs in random.h for some of the changes in bitcoin#17270. ACKs for top commit: practicalswift: ACK 55b2cb1 laanwj: ACK 55b2cb1 sipa: ACK 55b2cb1 Tree-SHA512: 672d369796e7c4f9b4d98dc545e5454999fa1bef373871994a26041d6163c58909e2255e4f820d3ef011679aa3392754eb57477306a89f5fd3d57e2bd7f0811a
Currently they are identical, but in the future we might want to turn the mempool into a unique_ptr. Replacing the global with the mempool pointer from the node context simplifies this step.
…ndonconflict dddd09e test: Wait until mempool is loaded in wallet_abandonconflict (MarcoFalke) Pull request description: This might or might not fix intermittent issues such as https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/28724018#L4091 I believe the mempool was not loaded fully after the restart, in which case it was not dumped either on the next restart. Thus, the previous mempool was attempted to be loaded a second time, which succeeded and contained the txs. ACKs for top commit: laanwj: ACK dddd09e Tree-SHA512: ab7061f946b5e5388f825dddceadb125f5197b24af3a7fcf1e700235d106a323419a56bfb4d84a2e27442e0de63e540c623b704343d83a98deaab3c02fcbdcbe
8bda096 Move events_hasher into RNGState() (Pieter Wuille) Pull request description: This moves `events_hasher` and `events_mutex` into `RNGState()` in random.cpp. This guarantees (through the existing `GetRNGState()` function) that the mutex is always created before any events are added, even when that happens inside global initializers. Fixes the issue reported here: bitcoin#17573 (comment), and includes the annotation from bitcoin#17666). ACKs for top commit: MarcoFalke: re-ACK 8bda096 🥈 sipsorcery: re-ACK 8bda096. Tree-SHA512: 78702d668764df19e9d61d87d82eca71cceca87d5351b740e13e732a1c18a3d53d7fbaaf63245266da597370bfebec9fa6a4749c15ec5a78dcfe6122c33553ed
… in Travis 1f9d5af tests: Add initialization order fiasco detection in Travis (practicalswift) Pull request description: Add initialization order fiasco detection in Travis :) Context: bitcoin#17670 (comment) This would have caught the `events_hasher` initialization order issue introduced in bitcoin#17573 and fixed in bitcoin#17670. Output in case of an initialization order fiasco: ``` ==7934==ERROR: AddressSanitizer: initialization-order-fiasco on address 0x557098d79200 at pc 0x55709796b9a3 bp 0x7ffde524dc30 sp 0x7ffde524dc28 READ of size 8 at 0x557098d79200 thread T0 #0 0x55709796b9a2 in CSHA256::Finalize(unsigned char*) src/crypto/sha256.cpp:667:25 #1 0x5570978150e9 in SeedEvents(CSHA512&) src/random.cpp:462:19 #2 0x5570978145e1 in SeedSlow(CSHA512&) src/random.cpp:482:5 bitcoin#3 0x5570978149a3 in SeedStartup(CSHA512&, (anonymous namespace)::RNGState&) src/random.cpp:527:5 bitcoin#4 0x55709781102d in ProcRand(unsigned char*, int, RNGLevel) src/random.cpp:571:9 bitcoin#5 0x557097810d19 in GetRandBytes(unsigned char*, int) src/random.cpp:576:59 bitcoin#6 0x557096c2f9d5 in (anonymous namespace)::CSignatureCache::CSignatureCache() src/script/sigcache.cpp:34:9 bitcoin#7 0x557096511977 in __cxx_global_var_init.7 src/script/sigcache.cpp:67:24 bitcoin#8 0x5570965119f8 in _GLOBAL__sub_I_sigcache.cpp src/script/sigcache.cpp bitcoin#9 0x557097bba4ac in __libc_csu_init (src/bitcoind+0x18554ac) bitcoin#10 0x7f214b1c2b27 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:266 bitcoin#11 0x5570965347d9 in _start (src/bitcoind+0x1cf7d9) 0x557098d79200 is located 96 bytes inside of global variable 'events_hasher' defined in 'random.cpp:456:16' (0x557098d791a0) of size 104 registered at: #0 0x557096545dfd in __asan_register_globals compiler-rt/lib/asan/asan_globals.cpp:360:3 #1 0x557097817f8b in asan.module_ctor (src/bitcoind+0x14b2f8b) SUMMARY: AddressSanitizer: initialization-order-fiasco src/crypto/sha256.cpp:667:25 in CSHA256::Finalize(unsigned char*) ``` ACKs for top commit: promag: Tested ACK 1f9d5af, got MarcoFalke: ACK 1f9d5af 👔 Tree-SHA512: f24ac0a313df7549193bd7f4fcfdf9b72bdfc6a6ee31d0b08e6d0752e5108fbd532106b6c86377ae0641258c9adb4921872e5d9a0154c0284e03315e0777102c
the `-` is not a special symbol and should not have `%` in front of it.
bd44711 build: pass -dead_strip_dylibs to ld on macOS (fanquake) Pull request description: This strips some unused dylibs from bitcoin-qt. ```diff otool -L src/qt/bitcoin-qt /usr/lib/libSystem.B.dylib - /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation -/System/Library/Frameworks/Security.framework/Versions/A/Security /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL -/System/Library/Frameworks/AGL.framework/Versions/A/AGL /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon /usr/lib/libc++.1.dylib /System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO /usr/lib/libobjc.A.dylib ``` `AGL` - ObjC wrapper for OpenGL. `DiskArbitration` - mount/unmount notifications and events. `Security` - low level security operations, authentication services. From `man ld`: ``` Remove dylibs that are unreachable by the entry point or exported symbols. That is, suppresses the generation of load command commands for dylibs which supplied no symbols during the link. This option should not be used when linking against a dylib which is required at runtime for some indirect reason such as the dylib has an important initializer. ``` ACKs for top commit: theuni: ACK bd44711. Tree-SHA512: 9592ce2966d28cb6c58e01efd401f56a4baa5dc5be5313f4fe8454632b578608be65a23c8602772049cd4655a9cb020fdd40d6622a244c301920d8c3db43f99a
897849d tests: Add deserialization fuzzing harnesses (practicalswift) 16f0a18 tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus (practicalswift) Pull request description: Add deserialization fuzzing harnesses. **Testing this PR** Run: ``` $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined $ make $ contrib/devtools/test_fuzzing_harnesses.sh 'addr_info|block_file_info|block_filter|block_header|ext_key|ext_pub_key|fee_rate|flat_file|key_origin|merkle_block|mutable_transaction|out_point|partial_merkle_tree|partially_signed_transaction|prefilled_transaction|psbt_input|psbt_output|pub_key|script_deserialize|sub_net|tx_in' 10 ``` `test_fuzzing_harnesses.sh` can be found in PR bitcoin#17000. ACKs for top commit: laanwj: thanks, ACK 897849d Tree-SHA512: 5a270a3002cc23b725f7b35476a43777b2b00b4d089cc006372e2fcc7afa430afaa3c1430f778ae08fc53dd85a13e7bd2fab0449c319f676423226e189a417f6
Last update was in 2017. Updates tinyformat to upstream commit 705e3f4e1de922069bf715746d35bd2364b1f98f. Re-apply bitcoin core specific changes. No changes that affect our use, as far as I can see, but this gets rid of the gcc `-Wimplicit-fallthrough` warnings, at least.
…key address 0950245 IsUsedDestination should count any known single-key address (Gregory Sanders) Pull request description: This plugs the privacy leak detailed at bitcoin#17605, at least for the single-key case. ACKs for top commit: meshcollider: Code Review ACK 0950245 Tree-SHA512: e1d68281675f05072b3087171cba1df9416a69c9ccf70c72e8555e55eadda2d0fd339e5a894e3a3438ff94b9e3827fb19b8b701faade70c08756b19ff157ee0c
…sabled 091a876 Test watchonly wallet bumpfee with PSBT return (Gregory Sanders) e9b4f94 bumpfee: Return PSBT when wallet has privkeys disabled (Gregory Sanders) 75a5e47 Change bumpfee to use watch-only funds for legacy watchonly wallets (Gregory Sanders) Pull request description: The main use-case here is for using with watch-only wallets with PSBT-signing cold wallets of all kinds. ACKs for top commit: achow101: ACK 091a876 Sjors: Tested ACK 091a876 meshcollider: utACK 091a876 Tree-SHA512: f7cf663e1af0b029e5c99eac88c5fdc3bc9e9a3841da8a608e8a9957e9bcf6a78864b8c2706fcaf78a480ffe11badd80c4fad29f97c0bb929e0470fafda5c22e
If QSettings is set already, it is required to force set nPruneSize after the intro dialog.
This function now resets the prune size and is only called after the intro sequence.
…revious behavior 8925df8 doc: update release notes (Jon Atack) 8bb405b test: getaddressinfo labels purpose deprecation test (Jon Atack) 60aba1f rpc: simplify getaddressinfo labels, deprecate previous behavior (Jon Atack) 7851f14 rpc: incorporate review feedback from PR 17283 (Jon Atack) Pull request description: This PR builds on bitcoin#17283 (now merged) and is followed by bitcoin#17585. It modifies the value returned by rpc getaddressinfo `labels` to an array of label name strings and deprecates the previous behavior of returning an array of JSON hash structures containing label `name` and address `purpose` key/value pairs. before ``` "labels": [ { "name": "DOUBLE SPEND", "purpose": "receive" } ``` after ``` "labels": [ "DOUBLE SPEND" ] ``` The deprecated behavior can be re-enabled by starting bitcoind with `-deprecatedrpc=labelspurpose`. For context, see: - bitcoin#17283 (comment) - http://www.erisian.com.au/bitcoin-core-dev/log-2019-12-13.html#l-425 (lines 425-427) - http://www.erisian.com.au/bitcoin-core-dev/log-2019-11-22.html#l-622 Reviewers: This PR may be tested manually by building, then running bitcoind with and without the `-deprecatedrpc=labelspurpose` flag while verifying the rpc getaddressinfo help text and `labels` output. Next steps: deprecate the rpc getaddressinfo `label` field (EDIT: done in bitcoin#17585) and add support for multiple labels per address. This PR will unblock those. ACKs for top commit: jnewbery: reACK 8925df8 promag: Code review ACK 8925df8. meshcollider: Code review ACK 8925df8 Tree-SHA512: c2b717209996da32b6484de7bb8800e7048410f9ce6afdb3e02a6866bd4a8f2c730f905fca27b10b877b91cf407f546e69e8c4feb9cd934325a6c71c166bd438
…M only e1e1442 Activate no-privkey -> ISMINE_WATCH_ONLY behavior for LegacySPKM only (Gregory Sanders) Pull request description: Slight cleanup following bitcoin#16944 This should allow future scriptpubkeymans to transparently work, since the current plan is to have ismine always be spendable. ACKs for top commit: achow101: ACK e1e1442 Sjors: Code review ACK e1e1442 meshcollider: Code review ACK e1e1442 Tree-SHA512: c0a86587d33b8b1646494a5cb0bf8681ee4a88e6913918157746943a0996b501903e0e6ee954cf04154c1e0faee0cbb375c74ca789f46ba9244eb5296632b042
77ef48d gitignore: ignore fuzz binaries, remove test_bitcoin_fuzzy (Jon Atack) Pull request description: The fuzzing gitignores haven't been updated since a4153e2 in 2016 that added an initial simple fuzzing framework. This commit: - removes `src/test/test_bitcoin_fuzzy` which is no longer used in favor of `src/test/fuzz` - ignores the `src/test/fuzz` directory, then un-ignores files in it with an extension, to de-clutter the git status from all the generated binary files. Co-authored-by: Karl-Johan Alm <karljohan-alm@garage.co.jp> ACKs for top commit: practicalswift: ACK 77ef48d MarcoFalke: ACK 77ef48d Tree-SHA512: 1fef4fbe88ded1ecf039205ace0da2abbaabcaae6ac6674eb29f84ae2f2fc560c1341e75e664cc2e243aca5767253756dba73b90ef32c7dd07c7c638abe6daf0
63bf06a Restore English translation option (Andrew Chow) Pull request description: It was [reported on Bitcointalk](https://bitcointalk.org/index.php?topic=5204167.msg53540137#msg53540137) that the normal English language option was lost in 0.19. This PR restores it. For some reason it was removed during the last periodic translation update. ACKs for top commit: laanwj: ACK 63bf06a Tree-SHA512: 94c7c7407f69e8df91fbbd8f8c5e3e8e031d308b72d775a00bcee564f2762a92f65c140029ce805faccdb767a25c0e222a396708c6ce29a5882bab939a45b772
a004673 qt: Add LogQtInfo() function (Hennadii Stepanov) Pull request description: This PR adds some info to `debug.log` I found useful for testing (e.g., on Wayland) and debugging issues like bitcoin#17153: ``` $ ./src/qt/bitcoin-qt -printtoconsole | head -n 6 2020-01-04T14:57:40Z [main] Bitcoin Core version v0.19.99.0-0df287f4e (release build) 2020-01-04T14:57:40Z [main] InitParameterInteraction: parameter interaction: -externalip set -> setting -discover=0 2020-01-04T14:57:40Z [main] Qt 5.9.5 (dynamic), plugin=xcb (dynamic) 2020-01-04T14:57:40Z [main] System: Linux Mint 19.3, x86_64-little_endian-lp64 2020-01-04T14:57:40Z [main] Screen: HDMI-1 1600x1200, pixel ratio=1.0 2020-01-04T14:57:40Z [main] Assuming ancestors of block 00000000000000b7ab6ce61eb6d571003fbe5fe892da4c9b740c49a07542462d have valid signatures. ``` ACKs for top commit: laanwj: ACK a004673 Tree-SHA512: 496bcfd4870a2730eab92b96b3e74989a7904b21369c372b6d4368f4ca2c141e2fdc1348a1fdd18cb68bb144dcea01d3023bb782f9d030e330c187f6a5a1a082
b0a2540 build: add Wdate-time to Werror flags (fanquake) Pull request description: `-Wdate-time` Warn when macros __TIME__, __DATE__ or __TIMESTAMP__ are encountered as they might prevent bit-wise-identical reproducible compilations. This is supported by [GCC](https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html) and [Clang](https://clang.llvm.org/docs/DiagnosticsReference.html#wdate-time). Example output: ```bash CXX bitcoind-bitcoind.o bitcoind.cpp:48:20: warning: expansion of date or time macro is not reproducible [-Wdate-time] printf("%s\n", __TIMESTAMP__); ^ bitcoind.cpp:49:20: warning: expansion of date or time macro is not reproducible [-Wdate-time] printf("%s\n", __TIME__); ^ bitcoind.cpp:50:20: warning: expansion of date or time macro is not reproducible [-Wdate-time] printf("%s\n", __DATE__); ^ 3 warnings generated. ``` ACKs for top commit: practicalswift: ACK b0a2540 -- diff looks correct and guarding against potential non-reproducibility is good :) promag: Tested ACK b0a2540 on macos with clang. Already had `--enable-werror`, added a wild `printf("%s\n", __TIMESTAMP__)` and got the following error: laanwj: ACK b0a2540 hebasto: ACK b0a2540 Tree-SHA512: b3a0b426e06dcd0c0baa94118c31158760b9690a8d0a15b5a2d544cb0879522e02817e134ef7346c707de09719818fc7e4bad1b3ad6b2dfe5e3c4169cdf5cb0d
fa37e0a test: Show debug log on unit test failure (MarcoFalke) Pull request description: Often, it is hard to debug unit test failures without the debug log. Especially when the failure happens remotely (e.g. on a ci system). Fix that by printing the log on failure. ACKs for top commit: jamesob: ACK fa37e0a ([`jamesob/ackr/16975.1.MarcoFalke.test_show_debug_log_on_u`](https://github.com/jamesob/bitcoin/tree/ackr/16975.1.MarcoFalke.test_show_debug_log_on_u)) Tree-SHA512: 2ca4150c4ae3d4ad47e03b5e5e70da2baffec928ddef1fdf53a3ebc061f14aee249205387cb1b12ef6d4eb55711ef0080c0b41d9d18000b5da124ca80299793b
…intro dialog af112ab qt: Rename SetPrune() to InitializePruneSetting() (Hennadii Stepanov) b0bfbe5 refactor: Drop `bool force' parameter (Hennadii Stepanov) 68c9bbe qt: Force set nPruneSize in QSettings after intro (Hennadii Stepanov) a82bd8f util: Replace magics with DEFAULT_PRUNE_TARGET_GB (Hennadii Stepanov) Pull request description: On master (5622d8f), having `QSettings` set already ``` $ grep nPruneSize ~/.config/Bitcoin/Bitcoin-Qt-testnet.conf nPruneSize=6 ``` enabling prune option in the intro dialog ``` $ ./src/qt/bitcoin-qt -choosedatadir -testnet ```  has no effect: ``` $ grep Prune ~/.bitcoin/testnet3/debug.log 2019-12-08T10:04:41Z Prune configured to target 5722 MiB on disk for block and undo files. ``` --- With this PR: ``` $ grep Prune ~/.bitcoin/testnet3/debug.log 2019-12-08T10:20:35Z Prune configured to target 1907 MiB on disk for block and undo files. ``` This PR has been split of bitcoin#17453 (the first two commits) as it fixes an orthogonal bug. Refs: - bitcoin#17453 (comment) - bitcoin#17453 (comment) ACKs for top commit: Sjors: Code review re-ACK af112ab ryanofsky: Code review ACK af112ab. Just suggested changes since last review (thanks!) promag: Tested ACK af112ab. Latest suggestions and changes look good to me. Tree-SHA512: 8ddad34b30dcc2cdcad6678ba8a0b36fa176e4e3465862ef6eee9be0f98d8146705138c9c7995dd8c0990af41078ca743fef1a90ed9240081f052f32ddec72b9
…tifiers 3e730bf zmq: Fix due to invalid argument and multiple notifiers (João Barbosa) Pull request description: ZMQ initialization is interrupted if any notifier fails, and in that case all notifiers are destroyed. The notifier shutdown assumes that the initialization had occurred. This is not valid when there are multiple notifiers and any except the last fails to initialize. Can be tested by running test/functional/interface_zmq.py from this branch with bitcoind from master. Closes bitcoin#17185. ACKs for top commit: laanwj: Code review ACK 3e730bf, thanks for adding a test Tree-SHA512: 5da710e97dcbaa94896d019e75162d470f6d381ee07c60e5b3e9db93d11e8f7ca9bf2c509efa4486199e88c96c3e720cc96b4e35b62725d4c7db8e8e9bf6e09d
6d6a7a8 gui: Fix duplicate wallet showing up (João Barbosa) 81ea66c Drop signal CClientUIInterface::LoadWallet (Russell Yanofsky) Pull request description: This PR includes 2 fixes: - prevent GUI LoadWallet handlers from crashing on startup when multiple handlers are attached, because the first handler takes ownership of the wallet unique pointer. Now every handler will receive its own unique pointer; - prevent showing a wallet twice in the GUI on startup due to a race with `loadwallet`. Fixes bitcoin#16937 ACKs for top commit: fjahr: code review ACK 6d6a7a8 ryanofsky: Code review ACK 6d6a7a8. No changes since last ACK other than rebase due to bitcoin#17070 kallewoof: Code review ACK 6d6a7a8 Tree-SHA512: 7f0658c9011f81dfa176a094c2263448ee1d14fda7dc94e8b55ee9c8b81538bd2d1e4bf8a8dbfcd029ebfc9feb6d3cda9dee3f911122df0a4b1e0ca75f653ba4
The UtxoSetHash is implemented as an index (subclass of BaseIndex) and maintains data in LevelDB. It keeps the muhash for each block in the database and also allows to search them by block hash or height. It also maintains the global Muhash object which is being modified as new blocks get added to the chain or as reorgs happen. This commit also adds a serialization method to the Muhash object to be able to persist it between restarts of the node.
fjahr
force-pushed
the
utxo-stats-index
branch
from
c7c9dd2
to
2a2c3c4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Todos