chore: merge from main #637
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chore(release): publish v1.6.0
## [1.6.0](v1.5.1...v1.6.0) (2024-05-09) ### Features * add Bitcoin transaction index to typescript client ([#568](#568)) ([6f7eba4](6f7eba4)) ### Bug Fixes * add stacks event position to ts client ([#575](#575)) ([3c48eeb](3c48eeb)) * add starting stacks height for prometheus metrics ([#567](#567)) ([6a8c086](6a8c086)) * make bitcoin payload serialization deserializable ([#569](#569)) ([5f20a86](5f20a86)) * set `Interrupted` status for streaming predicates that fail ([#574](#574)) ([11bde53](11bde53)), closes [#523](#523) * shut down observer on bitcoin block download failure ([#573](#573)) ([f3530b7](f3530b7)), closes [#572](#572)
Co-authored-by: Ludo Galabru <ludovic@galabru.com> Co-authored-by: Micaiah Reid <micaiahreid@gmail.com>
Previously, these were output as `CRIT` errors, but they should just be printed to the console.
The predicate status is used to report data to users and to pick back up where the service left off on a restart. This change updates the scanning predicate status to be updated on every successful trigger of a scanning predicate, to be sure we don't scan blocks twice on a restart (if the status was set to a previous block, we'd start on that block at startup)
Previously, when Chainhook was run as a service and the runloop to scan stacks/bitcoin predicates was set, we had no way to abort that scan. If a predicate was set to scan 1m blocks, but the user discovered the predicate was wrong and needed to delete, the user could delete the predicate from the store, but the scan thread had already started and would run until completion. This PR adds an abort signal to an ongoing scan so that when a predicate is deregistered, the scan is canceled.
### Description The goal of this PR is to make it much easier to use the Chainhook SDK. Previously, there were many fields that are rarely needed for the average user which had to be set when configuring the SDK. Many of these fields had confusing names that made it difficult to know how they were used in the SDK. Additionally, many of these fields were only needed for observing stacks events, but bitcoin-only users had to specify them anyways. This has a few major changes to the Chainhook SDK: - Removing some unused fields from the event observer config (`cache_path`, `data_handler_tx`, and`ingestion_port`) (694fb4d) - Renames `display_logs` -> `display_stacks_ingestion_logs` (694fb4d) - Renames `EventObserverConfigOverrides` -> `StacksEventObserverConfigBuilder` (9da1178) - Renames `ingestion_port` -> `chainhook_stacks_block_ingestion_port` for `StacksEventObserverConfigBuilder` (4e997fc) - Adds a `BitcoinEventObserverConfigBuilder` (fc67dff) - Renames some very confusingly named structs (5a4cb39): - `*ChainhookFullSpecification` => `*ChainhookSpecificationNetworkMap` - `*ChainhookNetworkSpecification` => `*ChainhookSpecification` - `*ChainhookSpecification` => `*ChainhookInstance` - refactor: moves stacks/bitcoin specific types to their respective types folder (83e8336) - adds helpers for registering chainhooks (4debc28) - renames `ChainhookConfig` -> `ChainhookStore` (c54b6e7) - add `EventObserverBuilder` to make a clean interface for starting an event observer (fe04dd9) - add a bunch of rsdoc comments with examples #### Breaking change? This will break some aspects of the Chainhook SDK. It should be a simple upgrade: - If you're building any of the above structs directly, rename the fields accordingly - If you're using `::new(..)` to build any of the above structs with fields that are removed, you may need to remove some fields - You can probably remove a good bit of code by using the builders ### Example New code example to start a bitcoin event observer: ```rust fn start_observer(ctx: &Context) -> Result<(), String> { let json_predicate = std::fs::read_to_string("./predicate.json").expect("Unable to read file"); let hook_instance: BitcoinChainhookInstance = serde_json::from_str(&json_predicate).expect("unable to parse chainhook spec"); let config = BitcoinEventObserverConfigBuilder::new() .rpc_username("regtest") .rpc_password("test1235") .rpc_url("http://0.0.0.0:8332") .finish()? .register_bitcoin_chainhook_instance(hook_instance)? .to_owned(); let (observer_commands_tx, observer_commands_rx) = channel(); EventObserverBuilder::new(config, &observer_commands_tx, observer_commands_rx, &ctx) .start() .map_err(|e| format!("event observer error: {}", e.to_string())) } ``` <details> <summary>Previous usage of starting a bitcoin observer</summary> ```rust let json_predicate = std::fs::read_to_string("./predicate.json").expect("Unable to read file"); let hook_spec: BitcoinChainhookFullSpecification = serde_json::from_str(&json_predicate).expect("unable to parse chainhook spec"); let bitcoin_network = BitcoinNetwork::Regtest; let stacks_network = chainhook_sdk::types::StacksNetwork::Mainnet; let mut bitcoin_hook_spec = hook_spec .into_selected_network_specification(&bitcoin_network) .expect("unable to parse bitcoin spec"); bitcoin_hook_spec.enabled = true; let mut chainhook_config = ChainhookConfig::new(); chainhook_config .register_specification(ChainhookSpecification::Bitcoin(bitcoin_hook_spec)) .expect("failed to register chainhook spec"); let config = EventObserverConfig { chainhook_config: Some(chainhook_config), bitcoin_rpc_proxy_enabled: false, ingestion_port: 0, bitcoind_rpc_username: "regtest".to_string(), bitcoind_rpc_password: "test1235".to_string(), bitcoind_rpc_url: "http://0.0.0.0:8332".to_string(), bitcoin_block_signaling: BitcoinBlockSignaling::ZeroMQ("tcp://0.0.0.0:18543".to_string()), display_logs: true, cache_path: String::new(), bitcoin_network: bitcoin_network, stacks_network: stacks_network, data_handler_tx: None, prometheus_monitoring_port: None, }; let (observer_commands_tx, observer_commands_rx) = channel(); // set up context to configure how we display logs from the event observer let logger = hiro_system_kit::log::setup_logger(); let _guard = hiro_system_kit::log::setup_global_logger(logger.clone()); let ctx = chainhook_sdk::utils::Context { logger: Some(logger), tracer: false, }; let moved_ctx = ctx.clone(); let _ = hiro_system_kit::thread_named("Chainhook event observer") .spawn(move || { let future = start_bitcoin_event_observer( config, observer_commands_tx, observer_commands_rx, None, None, moved_ctx, ); match hiro_system_kit::nestable_block_on(future) { Ok(_) => {} Err(e) => { println!("{}", e) } } }) .expect("unable to spawn thread"); ``` </details> Fixes #598
### Description
This PR adds validation for predicates. The
`ChainhookSpecificationNetworkMap::validate(&self)` method will now
check each of the fields in a predicate that could have invalid data and
return a string with _all_ of the errors separated by a `"\n"`.
I'm open to other ways of formatting the returned errors, but I think it
will be nice for users to see _everything_ that is wrong with their spec
in the first use rather than being given just the first error.
### Example
Here is an example result:
```
invalid Stacks predicate 'predicate_name' for network simnet: invalid 'then_that' value: invalid 'http_post' data: url string must be a valid Url: relative URL without a base
invalid Stacks predicate 'predicate_name' for network simnet: invalid 'then_that' value: invalid 'http_post' data: auth header must be a valid header value: failed to parse header value
invalid Stacks predicate 'predicate_name' for network simnet: invalid 'if_this' value: invalid predicate for scope 'print_event': invalid contract identifier: ParseError("Invalid principal literal: base58ck checksum 0x147e6835 does not match expected 0x9b3dfe6a")
```
---
### Checklist
- [x] All tests pass
- [x] Tests added in this PR (if applicable)
chore(release): publish v1.7.0
…dependency refactor: patch stacks-codec dependency
chore: update cargo lock
fix: typos
fix typo on monitoring
chore: update cargo lock to use the latest clarity-vm version removing lots of dependencies
closes #547 remove redundant PoxConfig and update uses to the chainhooks::types version --------- Co-authored-by: brady.ouren <bouren@hiro.sh>
chore: upgrade tokio and reqwest dependencies
refactor: use clarity crate
rafaelcr
had a problem deploying
to
Development-mainnet
GitHub Actions
Failure
rafaelcr
had a problem deploying
to
Development-testnet
GitHub Actions
Failure
tippenein
approved these changes
Aug 12, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.