Indexer shuts down node if prune mode is disabled?

[tzarebczan]

Indexer was still running, but full node was shutdown

2020-12-15T00:09:40.945Z	WARN	pruner	indexer/indexer.go:306	unable to prune lbrycrdd	{"prune height": 868152, "error": "invalid response: 500 Internal Server Error {\"result\":null,\"error\":{\"code\":-1,\"message\":\"Cannot prune blocks because node is not in prune mode.\"},\"id\":1}\n: error pruning blockchain"}
github.com/lbryio/rosetta-lbry/indexer.(*Indexer).Prune
	/app/src/indexer/indexer.go:306
main.startOnlineDependencies.func3
	/app/src/main.go:108
golang.org/x/sync/errgroup.(*Group).Go.func1
	/go/pkg/mod/golang.org/x/sync@v0.0.0-20201020160332-67f06af15bc9/errgroup/errgroup.go:57
2020-12-15T00:09:57.558Z	DEBUG	lbrycrdd	lbry/node.go:56	UpdateTip: new best=d66047065472d35f8cdf561b9b652c51d654f762a46fb06ec7828b521b1c4cf8 height=878723 version=0x20000000 log2_work=74.026143 txb=151 tx=26229131 date='2020-12-12T17:54:19Z' progress=0.182868 cache=956.5MiB(2445908txo) IBD
2020-12-15T00:10:13.007Z	DEBUG	lbrycrdd	lbry/node.go:56	UpdateTip: new best=c700e808aa053f8b3254e20b859b80137164e4e4d3c81c8f1f34c9e3ba683422 height=878730 version=0x20000000 log2_work=74.026155 txb=167 tx=26230115 date='2020-12-12T18:04:45Z' progress=0.183342 cache=957.5MiB(2449716txo) IBD
2020-12-15T00:10:29.568Z	DEBUG	lbrycrdd	lbry/node.go:56	UpdateTip: new best=7d764d3b11ecb9d62b745b1eb60d8ad57e327f00d07de4a0cba85a25f0e88547 height=878738 version=0x20000000 log2_work=74.026171 txb=11 tx=26231889 date='2020-12-12T18:24:08Z' progress=0.184238 cache=958.8MiB(2453162txo) IBD
2020-12-15T00:10:49.008Z	DEBUG	lbrycrdd	lbry/node.go:56	UpdateTip: new best=99d6e11b5f4ebd6bb3a4c2211b36f82c93ff51979b2c2ba5c61d8c291c15a8bf height=878744 version=0x20000000 log2_work=74.026184 txb=728 tx=26234057 date='2020-12-12T18:46:58Z' progress=0.185305 cache=960.2MiB(2456898txo) IBD
2020-12-15T00:10:52.745Z	DEBUG	memory	utils/utils.go:62	stats	{"heap (MB)": 3010.34708404541, "max heap (MB)": 8235.030403137207, "stack (MB)": 1.375, "system (MB)": 12090.720947265625, "garbage collections": 245}
2020-12-15T00:10:57.140Z	WARN	pruner	indexer/indexer.go:286	exiting pruner
github.com/lbryio/rosetta-lbry/indexer.(*Indexer).Prune
	/app/src/indexer/indexer.go:286
main.startOnlineDependencies.func3
	/app/src/main.go:108
golang.org/x/sync/errgroup.(*Group).Go.func1
	/go/pkg/mod/golang.org/x/sync@v0.0.0-20201020160332-67f06af15bc9/errgroup/errgroup.go:57
2020-12-15T00:10:57.141Z	WARN	lbrycrdd	lbry/node.go:98	sending interrupt to lbrycrdd
github.com/lbryio/rosetta-lbry/lbry.Startlbrycrdd.func3
	/app/src/lbry/node.go:98
golang.org/x/sync/errgroup.(*Group).Go.func1
	/go/pkg/mod/golang.org/x/sync@v0.0.0-20201020160332-67f06af15bc9/errgroup/errgroup.go:57
2020-12-15T00:10:57.194Z	DEBUG	lbrycrdd	lbry/node.go:56	tor: Thread interrupt
2020-12-15T00:10:57.215Z	DEBUG	lbrycrdd	lbry/node.go:56	addcon thread exit
2020-12-15T00:10:57.215Z	DEBUG	lbrycrdd	lbry/node.go:56	torcontrol thread exit
2020-12-15T00:10:57.215Z	DEBUG	lbrycrdd	lbry/node.go:56	opencon thread exit
2020-12-15T00:10:57.215Z	DEBUG	lbrycrdd	lbry/node.go:56	net thread exit
2020-12-15T00:10:57.220Z	DEBUG	lbrycrdd	lbry/node.go:56	Shutdown: In progress...
2020-12-15T00:10:58.229Z	DEBUG	lbrycrdd	lbry/node.go:56	msghand thread exit
2020-12-15T00:10:58.248Z	DEBUG	lbrycrdd	lbry/node.go:56	scheduler thread interrupt
2020-12-15T00:10:58.257Z	DEBUG	lbrycrdd	lbry/node.go:56	Dumped mempool: 0.002057s to copy, 0.001324s to dump

[patrick-ogrady]

Sorry for the confusion, @tzarebczan! Because rosetta-bitcoin stores all relevant data in its own indexer, we didn't at a mode to disable pruning because we didn't think anyone would want to keep a redundant copy of block data around.

If you want to disable pruning, you could:

1. Upstream a config option that disables the pruning goroutine
2. Remove the pruning goroutine in your fork

TL;DR...remove these 3 lines and it will work: https://github.com/coinbase/rosetta-bitcoin/blob/1b4c984e2d62c6c3bebc35d9810cf7963eef04e1/main.go#L107-L109

[patrick-ogrady]

@sidhujag asked about making this repo more generic so it is easier to use with bitcoin-core forks. Would that be something interesting to you? #50 (comment)

[sidhujag]

@sidhujag asked about making this repo more generic so it is easier to use with bitcoin-core forks. Would that be something interesting to you? #50 (comment)

I started to look at it but found a bunch of places that rely on genesis or network settings so I left it alone thinking you are probably better to do it @patrick-ogrady but I think it would be ideal. For us just the performance badger settings need to be set to be able to sync the rest is consistent (jsonify network and badger settings)

[patrick-ogrady]

I started to look at it but found a bunch of places that rely on genesis or network settings so I left it alone thinking you are probably better to do it @patrick-ogrady but I think it would be ideal.

👍

For us just the performance badger settings need to be set to be able to sync the rest is consistent (jsonify network and badger settings)

Yep, makes sense to me!