Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bitcoin depositor fee V2 to achieve better dev experience #2765

Merged
merged 9 commits into from
Aug 27, 2024

Conversation

ws4charlie
Copy link
Contributor

@ws4charlie ws4charlie commented Aug 23, 2024

Description

The current depositor fee calculation is based on the average fee rate of the block. After this modification, zetaclient will use inbound (the deposit tx) transaction's fee rate to calculate depositor fee.

This new impl will allow developer to accurately estimate depositor fee right after they build the deposit tx. The formula is

depositFee == (txFee / txVsize) * 68vB * 2

  • txFee is the deposit transaction's fee. This can be calculated by formula: totalInputValue - totalOutputValue
  • txVsize is the deposit transaction's size in vByte. This can be retrieved from RPC getrawtransaction (ZetaChain's implementation).

Closes: #2155

How Has This Been Tested?

  • Tested CCTX in localnet
  • Tested in development environment
  • Go unit tests
  • Go integration tests
  • Tested via GitHub Actions

Summary by CodeRabbit

  • New Features

    • Improved bitcoin depositor fee handling for enhanced accuracy and flexibility.
    • Added functionality for retrieving transaction details directly by hash.
    • Introduced new methods for calculating transaction fees and rates.
  • Bug Fixes

    • Enhanced error handling in fee calculations to ensure robust logging and management.
  • Documentation

    • Updated changelog to reflect the recent changes and feature additions related to bitcoin fees.
  • Tests

    • Refactored test suite for Bitcoin functionalities, improving clarity and modularity.
    • Added new tests for recently introduced fee calculation methods and transaction parsing.

Copy link
Contributor

coderabbitai bot commented Aug 23, 2024

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Walkthrough

The changes introduce a new feature for enhancing Bitcoin depositor fee calculations, allowing for more accurate fee assessments based on individual transaction parameters. This update includes new functions and constants to improve fee estimation and management within the Bitcoin chain client, alongside modifications to existing functions and enhancements to the testing framework.

Changes

Files Summary of Changes
changelog.md Added entry for the bitcoin depositor fee improvement feature.
zetaclient/chains/bitcoin/fee.go Introduced DynamicDepositorFeeHeightV2, added CalcDepositorFeeV2 and GetRecentFeeRate functions to enhance fee calculation and estimation.
zetaclient/chains/bitcoin/observer/inbound.go Updated GetBtcEvent to conditionally use CalcDepositorFeeV2 based on network conditions, with added error handling for fee calculation.
zetaclient/chains/bitcoin/observer/observer.go Removed rpc package import; updated specialHandleFeeRate to use bitcoin.GetRecentFeeRate instead of rpc.GetRecentFeeRate.
zetaclient/chains/bitcoin/rpc/rpc.go Removed constants related to fee rate estimation; added GetRawTxByHash and GetTransactionFeeAndRate functions for improved fee handling.
zetaclient/chains/bitcoin/rpc/rpc_live_test.go Restructured tests by removing BitcoinObserverTestSuite, refactoring into standalone functions, and improving error handling and naming conventions in test cases.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant ZetaHub
    participant BitcoinChainClient
    participant RPCClient

    User->>ZetaHub: Send Bitcoin transaction
    ZetaHub->>BitcoinChainClient: Calculate depositor fee
    BitcoinChainClient->>RPCClient: Get recent fee rate
    RPCClient-->>BitcoinChainClient: Return fee rate
    BitcoinChainClient-->>ZetaHub: Return calculated fee
    ZetaHub-->>User: Confirm transaction submission
Loading

Assessment against linked issues

Objective Addressed Explanation
Improve Bitcoin depositor fee calculation for better accuracy and user experience (#2155)
Enable detection of potential deposit failures in a timely manner.
Transition from average block fee to transaction-specific fee for deposit calculations.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

!!!WARNING!!!
nosec detected in the following files: zetaclient/chains/bitcoin/fee.go, zetaclient/chains/bitcoin/rpc/rpc_live_test.go

Be very careful about using #nosec in code. It can be a quick way to suppress security warnings and move forward with development, it should be employed with caution. Suppressing warnings with #nosec can hide potentially serious vulnerabilities. Only use #nosec when you're absolutely certain that the security issue is either a false positive or has been mitigated in another way.

Only suppress a single rule (or a specific set of rules) within a section of code, while continuing to scan for other problems. To do this, you can list the rule(s) to be suppressed within the #nosec annotation, e.g: /* #nosec G401 */ or //#nosec G201 G202 G203
Broad #nosec annotations should be avoided, as they can hide other vulnerabilities. The CI will block you from merging this PR until you remove #nosec annotations that do not target specific rules.

Pay extra attention to the way #nosec is being used in the files listed above.

@github-actions github-actions bot added the nosec label Aug 23, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (4)
zetaclient/chains/bitcoin/rpc/rpc.go (1)

57-69: Ensure comprehensive error handling in GetRawTxByHash.

The function GetRawTxByHash correctly wraps errors with additional context, which is good for debugging. However, consider logging the errors before returning them to provide more visibility into potential issues during runtime.

zetaclient/chains/bitcoin/fee.go (2)

40-44: Clarify the purpose of fee-related constants.

The constants DynamicDepositorFeeHeightV2, feeRateCountBackBlocks, and defaultTestnetFeeRate are introduced to manage fee calculations. Ensure that these values are documented and justified, especially in production environments, to avoid confusion.


273-311: Optimize error handling in GetRecentFeeRate.

The function retrieves the highest fee rate from recent blocks. Consider adding logging for each error case to improve traceability and debugging. Additionally, ensure that the default fee rate is appropriate for the testnet environment.

zetaclient/chains/bitcoin/rpc/rpc_live_test.go (1)

Line range hint 35-47: Add error handling for environment variables.

The function should handle cases where BTC_RPC_MAINNET or BTC_RPC_TESTNET environment variables are not set.

Consider adding checks and returning an error if these variables are missing.

 rpcMainnet := os.Getenv("BTC_RPC_MAINNET")
 rpcTestnet := os.Getenv("BTC_RPC_TESTNET")

+if rpcMainnet == "" || rpcTestnet == "" {
+    return nil, errors.New("RPC environment variables are not set")
+}
Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 1ad2bde and 625bc11.

Files selected for processing (6)
  • changelog.md (1 hunks)
  • zetaclient/chains/bitcoin/fee.go (3 hunks)
  • zetaclient/chains/bitcoin/observer/inbound.go (1 hunks)
  • zetaclient/chains/bitcoin/observer/observer.go (2 hunks)
  • zetaclient/chains/bitcoin/rpc/rpc.go (3 hunks)
  • zetaclient/chains/bitcoin/rpc/rpc_live_test.go (10 hunks)
Files skipped from review due to trivial changes (1)
  • changelog.md
Additional context used
Path-based instructions (5)
zetaclient/chains/bitcoin/rpc/rpc.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

zetaclient/chains/bitcoin/fee.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

zetaclient/chains/bitcoin/observer/inbound.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

zetaclient/chains/bitcoin/rpc/rpc_live_test.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

zetaclient/chains/bitcoin/observer/observer.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

Additional comments not posted (13)
zetaclient/chains/bitcoin/rpc/rpc.go (1)

126-169: Verify the fee calculation logic in GetTransactionFeeAndRate.

The function calculates transaction fees and rates based on input and output values. Ensure that the vin.Vout index is always valid and that the transaction structure aligns with expectations. Consider adding checks or logging to handle unexpected transaction structures.

Run the following script to verify the transaction structure:

zetaclient/chains/bitcoin/fee.go (1)

249-271: Verify the logic in CalcDepositorFeeV2.

The function calculates the depositor fee using the transaction's fee rate. Verify that the fee rate calculation aligns with the intended logic and that the BTCOutboundGasPriceMultiplier is correctly applied. Consider logging the calculated fee rate for debugging purposes.

Run the following script to verify the fee rate calculation:

Verification successful

Logic in CalcDepositorFeeV2 is Correctly Implemented

The function CalcDepositorFeeV2 correctly calculates the depositor fee by applying the BTCOutboundGasPriceMultiplier to the transaction's fee rate. The logic aligns with the intended design, and the multiplier is accurately set to 2.0. The existing tests confirm the correct application of this logic. No changes are necessary.

  • The fee rate is calculated using the transaction's input and output values.
  • The multiplier is correctly applied to the fee rate.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the fee rate calculation logic in `CalcDepositorFeeV2`.

# Test: Search for fee rate calculations. Expect: Correct application of BTCOutboundGasPriceMultiplier.
rg --type go -A 5 'CalcDepositorFeeV2'

Length of output: 3050


Script:

#!/bin/bash
# Verify the implementation of GetTransactionFeeAndRate and the value of BTCOutboundGasPriceMultiplier.

# Search for the implementation of GetTransactionFeeAndRate
ast-grep --lang go --pattern 'func GetTransactionFeeAndRate($_, $_) ($_, $_, error) { $$$ }'

# Search for the declaration of BTCOutboundGasPriceMultiplier
rg --type go 'BTCOutboundGasPriceMultiplier'

Length of output: 4641

zetaclient/chains/bitcoin/observer/inbound.go (1)

424-434: Ensure correctness of conditional logic in GetBtcEvent.

The conditional block switches to CalcDepositorFeeV2 based on network conditions. Verify that the conditions are correctly implemented and that the transition to version 2 fee calculation is seamless. Consider adding logging to track which fee calculation method is used.

Run the following script to verify the conditional logic:

Verification successful

Conditional Logic in GetBtcEvent Verified

The conditional logic for switching to CalcDepositorFeeV2 is correctly implemented based on network conditions and block height. The transition is seamless, and the function is well-tested in various scenarios. No changes are necessary.

  • The logic checks for TestNet3 or MainNet with the appropriate block height.
  • Extensive test coverage in inbound_test.go ensures robustness.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the conditional logic for switching to `CalcDepositorFeeV2`.

# Test: Search for conditional logic in `GetBtcEvent`. Expect: Correct implementation of network conditions.
rg --type go -A 5 'GetBtcEvent'

Length of output: 19792

zetaclient/chains/bitcoin/rpc/rpc_live_test.go (9)

Line range hint 62-79: LGTM!

The function correctly retrieves and handles the fee rate.


81-102: LGTM!

The function is well-structured and handles errors appropriately.


104-117: LGTM!

The function serves as a placeholder for running individual live tests.


119-154: LGTM!

The test function is comprehensive and checks multiple aspects of incoming transactions.


157-181: LGTM!

The test correctly verifies the absence of incoming transactions.


Line range hint 185-200: LGTM!

The test ensures proper creation and functionality of the Bitcoin RPC client.


Line range hint 204-222: LGTM!

The test effectively covers valid and invalid scenarios for block height retrieval.


Line range hint 226-272: LGTM!

The test effectively monitors and compares Bitcoin fee rates over time.


454-549: LGTM!

The test accurately compares calculated transaction fees and rates with external data.

zetaclient/chains/bitcoin/observer/observer.go (1)

Line range hint 630-634: LGTM!

The change to use bitcoin.GetRecentFeeRate simplifies the code and improves maintainability.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (5)
zetaclient/chains/bitcoin/rpc/rpc_live_test.go (4)

Line range hint 30-60: Add error handling for unset environment variables.

The createRPCClient function relies on environment variables for configuration. Consider adding checks to ensure these variables are set and provide meaningful error messages if they are not.

 rpcMainnet := os.Getenv("BTC_RPC_MAINNET")
 rpcTestnet := os.Getenv("BTC_RPC_TESTNET")

+ if rpcMainnet == "" || rpcTestnet == "" {
+   return nil, errors.New("environment variables BTC_RPC_MAINNET or BTC_RPC_TESTNET are not set")
+ }

Line range hint 185-202: Add assertions to verify RPC client properties.

Consider adding assertions to verify the properties of the created RPC client, such as connection status.

require.NotNil(t, client)
require.NoError(t, client.Ping())

Line range hint 226-274: Avoid infinite loops in tests.

The LiveTest_BitcoinFeeRate function contains an infinite loop, which is not suitable for tests. Consider using a finite loop or a timeout mechanism.

- for {
+ for i := 0; i < 5; i++ {

Line range hint 276-349: Consider using structured logging.

Using structured logging can improve the clarity and searchability of log messages.

log.Printf("block %d: gas rate %d == mempool.space gas rate\n", mb.Height, gasRate)
zetaclient/chains/bitcoin/observer/observer.go (1)

Line range hint 553-599: Enhance error messages for better debugging.

Consider adding more detailed error messages to aid in debugging.

 return "", errors.Wrapf(err, "error getting raw transaction %s", vin.Txid)
Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 1ad2bde and 625bc11.

Files selected for processing (6)
  • changelog.md (1 hunks)
  • zetaclient/chains/bitcoin/fee.go (3 hunks)
  • zetaclient/chains/bitcoin/observer/inbound.go (1 hunks)
  • zetaclient/chains/bitcoin/observer/observer.go (2 hunks)
  • zetaclient/chains/bitcoin/rpc/rpc.go (3 hunks)
  • zetaclient/chains/bitcoin/rpc/rpc_live_test.go (10 hunks)
Files skipped from review due to trivial changes (1)
  • changelog.md
Additional context used
Path-based instructions (5)
zetaclient/chains/bitcoin/rpc/rpc.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

zetaclient/chains/bitcoin/fee.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

zetaclient/chains/bitcoin/observer/inbound.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

zetaclient/chains/bitcoin/rpc/rpc_live_test.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

zetaclient/chains/bitcoin/observer/observer.go (1)

Pattern **/*.go: Review the Go code, point out issues relative to principles of clean code, expressiveness, and performance.

Additional comments not posted (18)
zetaclient/chains/bitcoin/rpc/rpc.go (2)

57-69: Function GetRawTxByHash is well-implemented.

The function correctly retrieves a raw transaction by hash and handles errors appropriately.


126-169: Function GetTransactionFeeAndRate is well-implemented.

The function accurately calculates transaction fees and rates, with appropriate error handling and safety checks.

zetaclient/chains/bitcoin/fee.go (3)

41-41: Constant DynamicDepositorFeeHeightV2 is well-defined.

The constant is clearly named and serves its intended purpose effectively.


249-271: Function CalcDepositorFeeV2 is well-implemented.

The function accurately calculates the depositor fee with appropriate error handling and network parameter checks.


273-311: Function GetRecentFeeRate is well-implemented.

The function effectively retrieves the highest fee rate from recent blocks with comprehensive error handling.

zetaclient/chains/bitcoin/observer/inbound.go (1)

424-435: Conditional logic in GetBtcEvent is well-implemented.

The function correctly determines when to use CalcDepositorFeeV2 based on network conditions and block height, with appropriate error handling.

zetaclient/chains/bitcoin/rpc/rpc_live_test.go (10)

Line range hint 62-79: LGTM! The function is well-implemented.

The getFeeRate function is correctly implemented with appropriate error handling.


104-117: LGTM! The function serves as a placeholder.

The Test_BitcoinLive function is appropriately set up as a placeholder for running individual live tests.


119-154: LGTM! The test is well-structured.

The LiveTest_FilterAndParseIncomingTx function is well-structured with clear assertions and error handling.


157-181: LGTM! The test correctly checks for no incoming transactions.

The LiveTest_FilterAndParseIncomingTx_Nop function is concise and correctly checks for the absence of incoming transactions.


Line range hint 204-222: LGTM! The test is well-implemented.

The LiveTest_GetBlockHeightByHash function is well-implemented with appropriate error handling and assertions.


Line range hint 351-362: LGTM! The test is straightforward and correct.

The LiveTest_AvgFeeRateMainnetMempoolSpace function is straightforward and correctly utilizes the helper function.


Line range hint 365-376: LGTM! The test is correctly implemented.

The LiveTest_AvgFeeRateTestnetMempoolSpace function is similar to the mainnet test and is correctly implemented.


379-387: LGTM! The test is concise and correct.

The LiveTest_GetRecentFeeRate function is concise and correctly asserts the fee rate.


454-549: LGTM! The test is comprehensive and detailed.

The Test_GetTransactionFeeAndRate function is comprehensive and includes detailed comparisons and assertions.


551-580: LGTM! The test is well-structured and meaningful.

The LiveTest_CalcDepositorFeeV2 function is well-structured and includes meaningful test cases.

zetaclient/chains/bitcoin/observer/observer.go (2)

Line range hint 72-116: LGTM! The function is well-implemented.

The NewObserver function is well-implemented, with appropriate error handling and initialization logic.


Line range hint 629-635: LGTM! The refactoring improves modularity.

The refactoring to use bitcoin.GetRecentFeeRate in specialHandleFeeRate improves modularity and maintainability.

Copy link

codecov bot commented Aug 23, 2024

Codecov Report

Attention: Patch coverage is 1.49254% with 66 lines in your changes missing coverage. Please review.

Project coverage is 66.78%. Comparing base (c3d1929) to head (3780897).
Report is 1 commits behind head on develop.

Files Patch % Lines
zetaclient/chains/bitcoin/fee.go 0.00% 30 Missing ⚠️
zetaclient/chains/bitcoin/rpc/rpc.go 0.00% 29 Missing ⚠️
zetaclient/chains/bitcoin/observer/inbound.go 20.00% 3 Missing and 1 partial ⚠️
zetaclient/chains/solana/signer/withdraw.go 0.00% 2 Missing ⚠️
zetaclient/chains/bitcoin/observer/observer.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2765      +/-   ##
===========================================
- Coverage    66.92%   66.78%   -0.15%     
===========================================
  Files          364      364              
  Lines        20471    20516      +45     
===========================================
+ Hits         13701    13702       +1     
- Misses        6143     6186      +43     
- Partials       627      628       +1     
Files Coverage Δ
zetaclient/chains/bitcoin/observer/observer.go 33.22% <0.00%> (ø)
zetaclient/chains/solana/signer/withdraw.go 0.00% <0.00%> (ø)
zetaclient/chains/bitcoin/observer/inbound.go 14.39% <20.00%> (+0.10%) ⬆️
zetaclient/chains/bitcoin/rpc/rpc.go 0.00% <0.00%> (ø)
zetaclient/chains/bitcoin/fee.go 37.98% <0.00%> (-11.52%) ⬇️

Copy link
Contributor

@skosito skosito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some comments here on PR that cherry picks this to release branch so please check there #2768

that PR should be updated it seems with this one

zetaclient/chains/bitcoin/fee.go Outdated Show resolved Hide resolved
@ws4charlie ws4charlie added this pull request to the merge queue Aug 27, 2024
Merged via the queue into develop with commit 36370df Aug 27, 2024
28 of 29 checks passed
@ws4charlie ws4charlie deleted the feat-btc-depositor-fee-V2 branch August 27, 2024 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants