feat: fast catchup #48

PhearZero · 2024-12-17T21:06:04Z

ℹ Overview

adds fast catchup
refactors internal library

Blocked by

feat: Node Install, Start, Stop, Upgrade ENG-572 #25

📝 Related Issues

relates to https://algorandfoundation.atlassian.net/browse/ENG-569

✅ Acceptance:

Pre-commit checks pass

codecov · 2024-12-17T21:34:46Z

Codecov Report

Attention: Patch coverage is 34.01970% with 737 lines in your changes missing coverage. Please review.

Project coverage is 38.05%. Comparing base (b38db59) to head (9d2f9b3).
Report is 20 commits behind head on main.

Files with missing lines	Patch %	Lines
api/lf.go	0.00%	214 Missing ⚠️
cmd/utils/config.go	31.81%	70 Missing and 5 partials ⚠️
internal/algod/state.go	40.44%	45 Missing and 8 partials ⚠️
cmd/node/catchup/debug.go	0.00%	44 Missing ⚠️
internal/algod/status.go	46.75%	34 Missing and 7 partials ⚠️
cmd/node/catchup/start.go	5.40%	35 Missing ⚠️
cmd/node/sync.go	0.00%	33 Missing ⚠️
cmd/node/catchup/stop.go	6.45%	29 Missing ⚠️
cmd/utils/flags.go	30.95%	25 Missing and 4 partials ⚠️
cmd/root.go	0.00%	27 Missing ⚠️
... and 25 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #48      +/-   ##
==========================================
- Coverage   38.55%   38.05%   -0.50%     
==========================================
  Files          72       73       +1     
  Lines        5514     5568      +54     
==========================================
- Hits         2126     2119       -7     
- Misses       3288     3345      +57     
- Partials      100      104       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

# This is the 1st commit message: feat: add prompt-ui # This is the commit message #2: feat: configure and set algod data directory # This is the commit message #3: fix: RX/TX display # This is the commit message #4: fix: bit rate display for GB # This is the commit message #5: fix: configuration override order # This is the commit message #6: feat: handle invalid configuration and token gracefully # This is the commit message #7: test: fix test state # This is the commit message #8: fix: loading of custom endpoint address # This is the commit message #9: fix: loading default port # This is the commit message #10: test: clear viper settings # This is the commit message #11: fix: finds path to directory and gives cmd instruction # This is the commit message #12: feat: adds node start and node stop commands # This is the commit message #13: fix: add -y # This is the commit message #14: fix: turn script into indivudal commands # This is the commit message #15: feat: check if sudo, clarify shell # This is the commit message #16: chore: make more go idiomatic # This is the commit message #17: fix: fix proper path check # This is the commit message #18: fix: interact with systemctl, cleanup prompts # This is the commit message #19: fix: remove sudo # This is the commit message #20: fix: separate commands # This is the commit message #21: fix: proper algorand service name # This is the commit message #22: fix: calling with sudo # This is the commit message #23: chore: testing systemctl # This is the commit message #24: fix: checks algorand system service has been enabled directly # This is the commit message #25: feat: implements editAlgorandServiceFile # This is the commit message #26: fix: else statement # This is the commit message #27: fix: quick check branch # This is the commit message #28: fix: string template # This is the commit message #29: feat: adds upgrade # This is the commit message #30: chore: removeu nnecessary code # This is the commit message #31: fix: check that installed and candidate are the same # This is the commit message #32: chore: improve print # This is the commit message #33: chore: add more output # This is the commit message #34: fix: single quote # This is the commit message #35: fix: -y # This is the commit message #36: fix: systemctl # This is the commit message #37: fix: upgrade and sudo text # This is the commit message #38: chore: go mod tidy # This is the commit message #39: fix: upgrade # This is the commit message #40: feat: disable ui elements while syncing # This is the commit message #41: feat: skip account loading on syncing feat: remove offline account expires date # This is the commit message #42: feat: installs algod and sets up service on mac # This is the commit message #43: feat: refactor, + mac # This is the commit message #44: feat: adds uninstall, mac only # This is the commit message #45: fix: remove plist file # This is the commit message #46: chore: rename # This is the commit message #47: test: protocol snapshots and 100% # This is the commit message #48: test: status snapshots and 100% # This is the commit message #49: test: error page snapshots and 100% # This is the commit message #50: test: controls snapshots # This is the commit message #51: test: accounts snapshots

# Conflicts: # cmd/root.go # internal/participation.go # internal/state.go # internal/status.go # ui/modal/controller.go # ui/modal/model.go # ui/modals/transaction/view.go

tasosbit · 2024-12-24T00:50:37Z

api/catchpoint.go

+	if err != nil {
+		return response, err
+	}
+	defer res.Body.Close()


should this be on line 64 instead?

Request body should be present on all non-nil errors.

tasosbit · 2024-12-24T01:01:16Z

cmd/node/catchup/stop.go

+	Short:        "Start a fast catchup",
+	Long:         "Checks if the node is caught up and if not, starts catching up.",


fix descriptions

tasosbit · 2024-12-24T01:03:59Z

cmd/node/catchup/stop.go

+		if err != nil {
+			log.Fatal(err)
+		}
+		log.Info(style.Green.Render("Latest Catchpoint: " + msg))


Does this make sense to print when we are stopping a catchpoint?

tasosbit · 2024-12-24T01:07:01Z

cmd/node/sync.go

+	style.Bold(syncCmdShortTxt),
+	"",
+	style.BoldUnderline("Overview:"),
+	"Fetch the latest catchpoint and use Fast-Catchup to check if the node is caught up to the latest block.",


This description indicates it is a status check (and not an action)

This one is technically both, since it is using the min parameter it should exit early if it is within the round window. It could have better verbiage to explain the early exit

tasosbit · 2024-12-24T01:18:53Z

internal/algod/participation/participation.go

 		if string(item.Key.VoteParticipationKey) == string(votekey) {
 			return &item.Id
 		}
 	}
 	return nil
 }

+// ToLoraDeepLink generates a Lora deep link URL for a transaction wizard based on network, offline state, and participation key.


are we using the toLoraDeepLink function anywhere now that we do the redirects?

No usages anymore, we can clean it up.

tasosbit · 2024-12-24T01:26:41Z

cmd/node/catchup/stop.go

+		if err != nil {
+			log.Fatal(err)
+		}
+		log.Info(style.Green.Render("Catchpoint Message: " + msg))


I think "Node response" or something similar works better. This usually returns "ok", right?

tasosbit

LGreatTM

# This is the 1st commit message: feat: add prompt-ui # This is the commit message #2: feat: configure and set algod data directory # This is the commit message #3: fix: RX/TX display # This is the commit message #4: fix: bit rate display for GB # This is the commit message #5: fix: configuration override order # This is the commit message #6: feat: handle invalid configuration and token gracefully # This is the commit message #7: test: fix test state # This is the commit message #8: fix: loading of custom endpoint address # This is the commit message #9: fix: loading default port # This is the commit message #10: test: clear viper settings # This is the commit message #11: fix: finds path to directory and gives cmd instruction # This is the commit message #12: feat: adds node start and node stop commands # This is the commit message #13: fix: add -y # This is the commit message #14: fix: turn script into indivudal commands # This is the commit message #15: feat: check if sudo, clarify shell # This is the commit message #16: chore: make more go idiomatic # This is the commit message #17: fix: fix proper path check # This is the commit message #18: fix: interact with systemctl, cleanup prompts # This is the commit message #19: fix: remove sudo # This is the commit message #20: fix: separate commands # This is the commit message #21: fix: proper algorand service name # This is the commit message #22: fix: calling with sudo # This is the commit message #23: chore: testing systemctl # This is the commit message #24: fix: checks algorand system service has been enabled directly # This is the commit message #25: feat: implements editAlgorandServiceFile # This is the commit message #26: fix: else statement # This is the commit message #27: fix: quick check branch # This is the commit message #28: fix: string template # This is the commit message #29: feat: adds upgrade # This is the commit message #30: chore: removeu nnecessary code # This is the commit message #31: fix: check that installed and candidate are the same # This is the commit message #32: chore: improve print # This is the commit message #33: chore: add more output # This is the commit message #34: fix: single quote # This is the commit message #35: fix: -y # This is the commit message #36: fix: systemctl # This is the commit message #37: fix: upgrade and sudo text # This is the commit message #38: chore: go mod tidy # This is the commit message #39: fix: upgrade # This is the commit message #40: feat: disable ui elements while syncing # This is the commit message #41: feat: skip account loading on syncing feat: remove offline account expires date # This is the commit message #42: feat: installs algod and sets up service on mac # This is the commit message #43: feat: refactor, + mac # This is the commit message #44: feat: adds uninstall, mac only # This is the commit message #45: fix: remove plist file # This is the commit message #46: chore: rename # This is the commit message #47: test: protocol snapshots and 100% # This is the commit message #48: test: status snapshots and 100% # This is the commit message #49: test: error page snapshots and 100% # This is the commit message #50: test: controls snapshots # This is the commit message #51: test: accounts snapshots

feat: fast catchup

PhearZero added 5 commits December 16, 2024 13:03

feat: add catchpoint rpc

05be82f

refactor: status and http interfaces

91e23f8

refactor: status, participation and state interfaces

911ad15

refactor: accounts and finalize state

f1be0a2

docs: add basic documentation

2d48fed

PhearZero added the feature label Dec 17, 2024

PhearZero added this to the Milestone 2 milestone Dec 17, 2024

PhearZero self-assigned this Dec 17, 2024

PhearZero force-pushed the feat/fast-catchup branch from 36ab7b0 to c8e1ff7 Compare December 17, 2024 21:24

PhearZero marked this pull request as ready for review December 17, 2024 21:35

chore: cleanup root commands

0f66c96

PhearZero force-pushed the feat/fast-catchup branch 2 times, most recently from b081bff to da3e55e Compare December 19, 2024 19:53

chore: polish commands and overviews

b38db59

PhearZero force-pushed the feat/fast-catchup branch from c3d3adc to b38db59 Compare December 19, 2024 20:09

Base automatically changed from feat/algod-node-install to main December 23, 2024 21:30

Merge branch 'main' into feat/fast-catchup

5acfe83

# Conflicts: # cmd/root.go # internal/participation.go # internal/state.go # internal/status.go # ui/modal/controller.go # ui/modal/model.go # ui/modals/transaction/view.go

tasosbit reviewed Dec 24, 2024

View reviewed changes

PhearZero added 2 commits December 23, 2024 20:09

chore: fix stop fast catchup verbiage

672b9ba

chore: fix info log message for catchup abort

9d0dba5

tasosbit reviewed Dec 24, 2024

View reviewed changes

chore: remove lora deep-link

9d2f9b3

tasosbit reviewed Dec 24, 2024

View reviewed changes

tasosbit approved these changes Dec 24, 2024

View reviewed changes

PhearZero merged commit 1ea642e into main Dec 24, 2024
4 of 5 checks passed

PhearZero deleted the feat/fast-catchup branch December 24, 2024 12:46

tasosbit pushed a commit that referenced this pull request Jan 22, 2025

Merge pull request #48 from algorandfoundation/feat/fast-catchup

22e239a

feat: fast catchup

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: fast catchup #48

feat: fast catchup #48

PhearZero commented Dec 17, 2024 •

edited

Loading

codecov bot commented Dec 17, 2024 •

edited

Loading

tasosbit Dec 24, 2024

PhearZero Dec 24, 2024

tasosbit Dec 24, 2024

tasosbit Dec 24, 2024

tasosbit Dec 24, 2024

PhearZero Dec 24, 2024 •

edited

Loading

tasosbit Dec 24, 2024

PhearZero Dec 24, 2024

tasosbit Dec 24, 2024

tasosbit left a comment

		Short: "Start a fast catchup",
		Long: "Checks if the node is caught up and if not, starts catching up.",

feat: fast catchup #48

feat: fast catchup #48

Conversation

PhearZero commented Dec 17, 2024 • edited Loading

ℹ Overview

Blocked by

📝 Related Issues

✅ Acceptance:

codecov bot commented Dec 17, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PhearZero Dec 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tasosbit left a comment

Choose a reason for hiding this comment

PhearZero commented Dec 17, 2024 •

edited

Loading

codecov bot commented Dec 17, 2024 •

edited

Loading

PhearZero Dec 24, 2024 •

edited

Loading