Skip to content

Conversation

@rustyrussell
Copy link
Contributor

It's very hard to reproduce, since we only consider UTXOs we are watching, but scanning from the first block we know about is wrong, because we don't care about blocks which we only put in the db in response to old gossip queries.

I'm not sure how Sjors got into the state where they see their own UTXO spend in a block they don't have in the database, but we shouldn't crash:

2025-12-07T11:25:18.163Z **BROKEN** lightningd: Error executing statement: wallet/wallet.c:4913: UPDATE outputs SET spend_height = ?,  status = ? WHERE prev_out_tx = ? AND prev_out_index = ?: FOREIGN KEY constraint failed
2025-12-07T11:25:18.163Z **BROKEN** lightningd: Error executing statement: wallet/wallet.c:4913: UPDATE outputs SET spend_height = ?,  status = ? WHERE prev_out_tx = ? AND prev_out_index = ?: FOREIGN KEY constraint failed
2025-12-07T11:25:18.179Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v25.12)
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: common/daemon.c:46 (send_backtrace) 0x5e263feb9292
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: common/daemon.c:83 (crashdump) 0x5e263feb92e1
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f64532f
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f69eb2c
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f64527d
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f6288fe
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/log.c:1054 (fatal_vfmt) 0x5e263fe38e10
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: wallet/db.c:95 (db_error) 0x5e263fe6f7ef
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: db/utils.c:326 (db_fatal) 0x5e263feaa797
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: db/utils.c:200 (db_exec_prepared_v2) 0x5e263feaab85
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: wallet/wallet.c:4924 (wallet_outpoint_spend) 0x5e263fe7de2f
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:929 (topo_update_spends) 0x5e263fe0af81
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:1532 (fixup_scan_block) 0x5e263fe0c268
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/bitcoind.c:503 (getrawblockbyheight_callback) 0x5e263fe09280
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:701 (plugin_response_handle) 0x5e263fe5bf2b
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:790 (plugin_read_json) 0x5e263fe610e7
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:60 (next_plan) 0x5e263fef3629
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:422 (do_plan) 0x5e263fef3afa
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:439 (io_ready) 0x5e263fef3bb7
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:470 (io_loop) 0x5e263fef55c5
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x5e263fe308e9
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1492 (main) 0x5e263fe363ce
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f62a1c9
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f62a28a
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x5e263fe06bf4
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff

Changelog-Fixes: lightningd: crash on fixup scan with old blocks.
Closes: #8765

@rustyrussell rustyrussell added this to the v26.03 milestone Dec 8, 2025
@rustyrussell rustyrussell added the 25.12.1 Point release for 25.12 label Dec 8, 2025
It's very hard to reproduce, since we only consider UTXOs we are watching, but scanning
from the first block we know about is wrong, because we don't care about blocks which
we only put in the db in response to old gossip queries.

I'm not sure how Sjors got into the state where they see their own UTXO spend in a block
they don't have in the database, but we shouldn't crash:

```
2025-12-07T11:25:18.163Z **BROKEN** lightningd: Error executing statement: wallet/wallet.c:4913: UPDATE outputs SET spend_height = ?,  status = ? WHERE prev_out_tx = ? AND prev_out_index = ?: FOREIGN KEY constraint failed
2025-12-07T11:25:18.163Z **BROKEN** lightningd: Error executing statement: wallet/wallet.c:4913: UPDATE outputs SET spend_height = ?,  status = ? WHERE prev_out_tx = ? AND prev_out_index = ?: FOREIGN KEY constraint failed
2025-12-07T11:25:18.179Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v25.12)
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: common/daemon.c:46 (send_backtrace) 0x5e263feb9292
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: common/daemon.c:83 (crashdump) 0x5e263feb92e1
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f64532f
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f69eb2c
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f64527d
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f6288fe
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/log.c:1054 (fatal_vfmt) 0x5e263fe38e10
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: wallet/db.c:95 (db_error) 0x5e263fe6f7ef
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: db/utils.c:326 (db_fatal) 0x5e263feaa797
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: db/utils.c:200 (db_exec_prepared_v2) 0x5e263feaab85
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: wallet/wallet.c:4924 (wallet_outpoint_spend) 0x5e263fe7de2f
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:929 (topo_update_spends) 0x5e263fe0af81
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/chaintopology.c:1532 (fixup_scan_block) 0x5e263fe0c268
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/bitcoind.c:503 (getrawblockbyheight_callback) 0x5e263fe09280
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:701 (plugin_response_handle) 0x5e263fe5bf2b
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:790 (plugin_read_json) 0x5e263fe610e7
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:60 (next_plan) 0x5e263fef3629
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:422 (do_plan) 0x5e263fef3afa
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:439 (io_ready) 0x5e263fef3bb7
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:470 (io_loop) 0x5e263fef55c5
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x5e263fe308e9
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1492 (main) 0x5e263fe363ce
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f62a1c9
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x70ee6f62a28a
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x5e263fe06bf4
2025-12-07T11:25:18.180Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
```

Changelog-Fixes: lightningd: crash on fixup scan with old blocks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

25.12.1 Point release for 25.12

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FOREIGN KEY constraint failed after v25.12 upgrade

1 participant