Skip to content

build: decouple secp256 tests from BUILD_TESTS#30

Draft
willcl-ark wants to merge 4 commits intomasterfrom
disable-secp-tests
Draft

build: decouple secp256 tests from BUILD_TESTS#30
willcl-ark wants to merge 4 commits intomasterfrom
disable-secp-tests

Conversation

@willcl-ark
Copy link
Owner

@willcl-ark willcl-ark commented Jun 19, 2025

.These tests add a minimum of a few minutes to the unit test runtime, and
at worst see runtime double.

Decouple them from ${BUILD_TESTS} so that they can be more selectively
enabled/disabled.

@willcl-ark
build: decouple secp256 tests from BUILD_TESTS
39591a3 
These tests add a minimum of a few minutes to the unit test runtime, and
at worst see runtime double.

Decouple them from ${BUILD_TESTS} so that they can be more selectively
enabled/disabled.
@willcl-ark willcl-ark force-pushed the disable-secp-tests branch from 7fe68fe to 5fa53f0 Compare June 19, 2025 13:53
@willcl-ark
ci: add secp256 unit test job
5d48bba 
This workflow job detects whether the secp256k1 subtree was updated,
and, if it was, will build and run the secp256k1 unit tests.

These are disable by default (in other CI jobs), as they doubel the
runtime needed, and rarely change.
@willcl-ark
optional: enable secp tests in dev mode
d519934 
We could choose to enable the secp256 tests in dev mode, but I'm not
convinced that it's even necessary here, unless the subtree changes.
@willcl-ark
DELETE: modify secp256
267ba87 
Dummy commit to show the new CI job running.
@willcl-ark willcl-ark force-pushed the disable-secp-tests branch from 5fa53f0 to 267ba87 Compare June 19, 2025 19:43
@willcl-ark willcl-ark force-pushed the master branch 24 times, most recently from 2fcab0a to b78454f Compare September 10, 2025 10:06
@github-actions github-actions bot force-pushed the master branch 3 times, most recently from 97ed557 to e472e80 Compare October 28, 2025 23:54
@willcl-ark willcl-ark force-pushed the master branch 2 times, most recently from 3108440 to 2370337 Compare October 30, 2025 10:31
@github-actions github-actions bot force-pushed the master branch 2 times, most recently from a5d742e to acc041c Compare October 31, 2025 23:54
@willcl-ark willcl-ark force-pushed the master branch 2 times, most recently from 37a6a6d to 4cdce02 Compare November 5, 2025 22:25
@github-actions github-actions bot force-pushed the master branch 4 times, most recently from 6a31544 to acff675 Compare November 11, 2025 23:54
@willcl-ark willcl-ark force-pushed the master branch 9 times, most recently from ed138e2 to 2ccfcf2 Compare November 20, 2025 23:55
@willcl-ark willcl-ark force-pushed the master branch 3 times, most recently from 425aeb2 to 9827b14 Compare November 28, 2025 12:03
@willcl-ark willcl-ark force-pushed the master branch 2 times, most recently from dd7a4bd to 521a43c Compare February 16, 2026 19:46
willcl-ark pushed a commit that referenced this pull request Feb 25, 2026
@achow101
Merge bitcoin#34655: fuzz: keep coins_view fuzzers within caller co…
76eb04b 
…ntracts

3281824 fuzz: prevent invalid `FRESH` entries and surface `BatchWrite` errors (Lőrinc)
780f460 fuzz: avoid invalid `AddCoin` overwrites (Lőrinc)
d7e0d51 fuzz: make `AddCoins` query view for overwrites (Lőrinc)
b8fa6f0 util: introduce `TrySub` to prevent unsigned underflow (Lőrinc)

Pull request description:

  ### Problem
  This is an alternative approach to bitcoin#34647, fixes bitcoin#34645.

  ### Fix
  First, add `CheckedSub` and use it for decrements of `m_dirty_count` and `cachedCoinsUsage`, so unsigned underflows turn into immediate failures instead of silently wrapping and only failing later.

  <details><summary>Assertion `j <= i' failed.</summary>

  ```bash
  util/overflow.h:44 T CheckedSub(const T, const U) [T = unsigned long, U = bool]: Assertion `j <= i' failed.
  ==72817== ERROR: libFuzzer: deadly signal
      #0 0x556e9225eab5 in __sanitizer_print_stack_trace (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x191dab5) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)
      #1 0x556e921acafc in fuzzer::PrintStackTrace() (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x186bafc) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)
      #2 0x556e92191bb7 in fuzzer::Fuzzer::CrashCallback() (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x1850bb7) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)
      #3 0x7164cfc458cf  (/lib/x86_64-linux-gnu/libc.so.6+0x458cf) (BuildId: ae7440bbdce614e0e79280c3b2e45b1df44e639c)
      #4 0x7164cfca49bb in __pthread_kill_implementation nptl/pthread_kill.c:43:17
      #5 0x7164cfca49bb in __pthread_kill_internal nptl/pthread_kill.c:89:10
      #6 0x7164cfca49bb in pthread_kill nptl/pthread_kill.c:100:10
      #7 0x7164cfc4579d in raise signal/../sysdeps/posix/raise.c:26:13
      bitcoin#8 0x7164cfc288cc in abort stdlib/abort.c:73:3
      #9 0x556e92f9d591 in assertion_fail(std::source_location const&, std::basic_string_view<char, std::char_traits<char>>) /mnt/my_storage/bitcoin/src/util/check.cpp:41:5
      #10 0x556e9250daf0 in bool&& inline_assertion_check<false, bool>(bool&&, std::source_location const&, std::basic_string_view<char, std::char_traits<char>>) /mnt/my_storage/bitcoin/src/util/check.h:90:13
      #11 0x556e9250daf0 in unsigned long CheckedSub<unsigned long, bool>(unsigned long, bool) /mnt/my_storage/bitcoin/src/util/overflow.h:44:5
      #12 0x556e9250daf0 in CoinsViewCacheCursor::NextAndMaybeErase(std::pair<COutPoint const, CCoinsCacheEntry>&) /mnt/my_storage/bitcoin/src/coins.h:282:25
      #13 0x556e92507eb2 in (anonymous namespace)::MutationGuardCoinsViewCache::BatchWrite(CoinsViewCacheCursor&, uint256 const&) /mnt/my_storage/bitcoin/src/test/fuzz/coins_view.cpp:90:75
      #14 0x556e92c17a2b in CCoinsViewCache::Flush(bool) /mnt/my_storage/bitcoin/src/coins.cpp:282:11
      #15 0x556e924fb732 in TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_1::operator()() const /mnt/my_storage/bitcoin/src/test/fuzz/coins_view.cpp:135:34
      #16 0x556e924fb732 in unsigned long CallOneOf<TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_0, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_1, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_2, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_3, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_4, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_5, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_6, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_7, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_8, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_9, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_10, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_11>(FuzzedDataProvider&, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_0, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_1, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_2, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_3, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_4, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_5, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_6, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_7, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_8, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_9, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_10, TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool)::$_11) /mnt/my_storage/bitcoin/src/test/fuzz/util.h:42:27
      #17 0x556e924fb732 in TestCoinsView(FuzzedDataProvider&, CCoinsViewCache&, CCoinsView&, bool) /mnt/my_storage/bitcoin/src/test/fuzz/coins_view.cpp:114:9
      #18 0x556e92503b0c in coins_view_overlay_fuzz_target(std::span<unsigned char const, 18446744073709551615ul>) /mnt/my_storage/bitcoin/src/test/fuzz/coins_view.cpp:404:5
      #19 0x556e92bcb7a5 in std::function<void (std::span<unsigned char const, 18446744073709551615ul>)>::operator()(std::span<unsigned char const, 18446744073709551615ul>) const /usr/lib/gcc/x86_64-linux-gnu/15/../../../../include/c++/15/bits/std_function.h:593:9
      #20 0x556e92bcb7a5 in test_one_input(std::span<unsigned char const, 18446744073709551615ul>) /mnt/my_storage/bitcoin/src/test/fuzz/fuzz.cpp:88:5
      #21 0x556e92bcb7a5 in LLVMFuzzerTestOneInput /mnt/my_storage/bitcoin/src/test/fuzz/fuzz.cpp:216:5
      #22 0x556e9219318f in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x185218f) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)
      #23 0x556e92192799 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x1851799) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)
      #24 0x556e92194139 in fuzzer::Fuzzer::MutateAndTestOne() (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x1853139) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)
      #25 0x556e92194c95 in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile>>&) (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x1853c95) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)
      #26 0x556e92181255 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x1840255) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)
      #27 0x556e921ad696 in main (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x186c696) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)
      #28 0x7164cfc2a577 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
      #29 0x7164cfc2a63a in __libc_start_main csu/../csu/libc-start.c:360:3
      #30 0x556e921757e4 in _start (/mnt/my_storage/bitcoin/build_fuzz/bin/fuzz+0x18347e4) (BuildId: d77c4d5f9dfd38ea06fab463f49341735205e109)

  NOTE: libFuzzer has rudimentary signal handlers.
        Combine libFuzzer with AddressSanitizer or similar for better crash reports.
  SUMMARY: libFuzzer: deadly signal
  MS: 2 PersAutoDict-CopyPart- DE: "\005\000"-; base unit: ecb626aff8724f0fdde38a0a6965718f2096d474
  artifact_prefix='/tmp/fuzz_artifacts/'; Test unit written to /tmp/fuzz_artifacts/crash-1d19026c1a23f08bfe693fd684a56ce51187c6e5
  ./build_fuzz/bin/fuzz /tmp/fuzz_corpus/coins_view_overlay -max_total_time=3600 -rss_limit_mb=2560 -artifact_prefix=/tmp/fuzz_artifacts/ >fuzz-16.log 2>&1
  ```

  </details>

  The coins view fuzz targets can call `AddCoin`/`AddCoins` and construct `BatchWrite` cursors in ways that violate `CCoinsViewCache` caller contracts. These invalid states can trigger `BatchWrite` `std::logic_error` and can desync dirty-entry accounting (caught by `Assume(m_dirty_count == 0)` currently).

  Make the fuzzer avoid generating invalid states instead of catching and resetting:
  * Derive `AddCoin`’s `possible_overwrite` from `PeekCoin`, so `possible_overwrite=false` is only used when the outpoint is absent - similarly to https://github.com/bitcoin/bitcoin/blob/67c0d1798e6147f48d4bafc2c9e5ff30f2a62340/src/test/fuzz/coinscache_sim.cpp#L312-L317
  - Only use `AddCoins(check=false)` when we have confirmed the txid has no unspent outputs; otherwise fall back to `check=true` so `AddCoins` determines overwrites via the view.
  - When constructing a `CoinsViewCacheCursor`, avoid setting `FRESH` when the parent already has an unspent coin, and ensure `FRESH` implies `DIRTY`.

  ### Fuzzing
  The original error could be reproduced in ~10 minutes using `coins_view_overlay`. I ran the `coins_view`, `coins_view_db`, `coins_view_overlay`, and `coinscache_sim` fuzzers for this PR overnight and they didn't fail anymore.

ACKs for top commit:
  achow101:
    ACK 3281824
  sipa:
    ACK 3281824. Ran the 4 relevant fuzz tests for ~1 CPU day each. Will run more overnight.
  andrewtoth:
    ACK 3281824

Tree-SHA512: b8155e8d21740eb7800e373c27a8a1457eb84468c24af879bac5a1ed251ade2aec99c34a350a31f2ebb74e41bb7380bf20214d38d14fe23310a43282d2434fb7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@willcl-ark