-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Loading status checks…
improving README.md
Showing
1 changed file
with
143 additions
and
59 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,100 +1,184 @@ | ||
|
|
||
| # dsh-sdk-platform-rs | ||
|
|
||
| [](https://github.com/kpn-dsh/dsh-sdk-platform-rs/actions/workflows/main.yaml) | ||
| [](https://codecov.io/gh/kpn-dsh/dsh-sdk-platform-rs) | ||
| [](https://deps.rs/repo/github/kpn-dsh/dsh-sdk-platform-rs) | ||
| [](https://opensource.org/licenses/Apache-2.0) | ||
|
|
||
| A Rust SDK to interact with the DSH Platform. This library provides convenient building blocks for services that need to connect to DSH Kafka, fetch tokens for various protocols, manage Prometheus metrics, and more. | ||
|
|
||
| > **Note** | ||
| > This library (v0.5.x) is a _release candidate_. It may contain incomplete features and/or bugs. Future updates might introduce breaking changes. Please report any issues you find. | ||
| --- | ||
|
|
||
| ## Table of Contents | ||
|
|
||
| 1. [Migration Guide 0.4.X -> 0.5.X](#migration-guide-04x---05x) | ||
| 2. [Description](#description) | ||
| 3. [Usage](#usage) | ||
| 4. [Connecting to DSH](#connect-to-dsh) | ||
| 5. [Feature Flags](#feature-flags) | ||
| 6. [Environment Variables](#environment-variables) | ||
| 7. [Examples](#examples) | ||
| 8. [Changelog](#changelog) | ||
| 9. [Contributing](#contributing) | ||
| 10. [License](#license) | ||
| 11. [Security](#security) | ||
|
|
||
| --- | ||
|
|
||
| ## Migration Guide 0.4.X -> 0.5.X | ||
|
|
||
| If you are migrating from `0.4.X` to `0.5.X`, please see the [migration guide](https://github.com/kpn-dsh/dsh-sdk-platform-rs/wiki/Migration-guide-(v0.4.X-%E2%80%90--v0.5.X)) for details on breaking changes and how to update your code accordingly. | ||
|
|
||
| --- | ||
|
|
||
| ## Description | ||
| This library can be used to interact with the DSH Platform. It is intended to be used as a base for services that will be used to interact with DSH. Features include: | ||
| - Connect to DSH | ||
| - Fetch Kafka Properties and certificates | ||
| - Rest API Token Fetcher (to be used with [dsh_rest_api_client](https://crates.io/crates/dsh_rest_api_client)) | ||
| - MQTT Token Fetcher | ||
| - Common functions | ||
| - Preconfigured RDKafka client config | ||
| - Preconfigured Reqwest client config (for schema store) | ||
| - Graceful shutdown | ||
| - Prometheus Metrics (web server and re-export of metrics crate) | ||
| - Dead Letter Queue (experimental) | ||
|
|
||
| ### Note | ||
| Rdkafka and thereby this library is dependent on CMAKE. Make sure it is installed in your environment and/or Dockerfile where you are compiling. | ||
| See [dockerfile](../example_dsh_service/Dockerfile) for an example. | ||
|
|
||
| The `dsh-sdk-platform-rs` library offers: | ||
|
|
||
| - **DSH Kafka Connectivity** | ||
| - Supports both direct DSH, Kafka Proxy, VPN, and local Kafka. | ||
| - Handles datastream information retrieval, certificate signing (bootstrap), and PKI configuration. | ||
|
|
||
| - **Token Fetchers** | ||
| - **Management API Token Fetcher**: For use with [`dsh_rest_api_client`](https://crates.io/crates/dsh_rest_api_client). | ||
| - **Protocol Token Fetcher**: Obtain tokens for MQTT and HTTP protocol adapters. | ||
|
|
||
| - **Kafka Configuration** | ||
| - RDKafka-based configuration, including vendored SSL support. | ||
|
|
||
| - **Common Utilities** | ||
| - Prometheus metrics (built-in HTTP server, plus re-export of the `metrics` crate). | ||
| - Tokio-based graceful shutdown handling. | ||
| - Dead Letter Queue (DLQ) functionality. | ||
|
|
||
| --- | ||
|
|
||
| ## Usage | ||
| To use this SDK with the default features in your project, add the following to your Cargo.toml file: | ||
|
|
||
| ```toml | ||
| [dependencies] | ||
| dsh_sdk = "0.4" | ||
| ``` | ||
|
|
||
| However, if you would like to use only specific features, you can specify them in your Cargo.toml file. For example, if you would like to use only the bootstrap feature, add the following to your Cargo.toml file: | ||
| To get started, add the following to your `Cargo.toml`: | ||
|
|
||
| ```toml | ||
| [dependencies] | ||
| dsh_sdk = { version = "0.4", default-features = false, features = ["bootstrap"] } | ||
| dsh_sdk = "0.5" | ||
| rdkafka = { version = "0.37", features = ["cmake-build", "ssl-vendored"] } | ||
| ``` | ||
|
|
||
| See [feature flags](#feature-flags) for more information on the available features. | ||
| > **Note** | ||
| > By default, this SDK enables several features (see [Feature Flags](#feature-flags)). If you do not need them all, you can disable default features to reduce compile times and dependencies. | ||
| ### Example | ||
|
|
||
| To use this SDK in your project | ||
| ```rust | ||
| use dsh_sdk::Properties; | ||
| use dsh_sdk::rdkafka::consumer::{Consumer, StreamConsumer}; | ||
| use dsh_sdk::DshKafkaConfig; // Trait for applying DSH-specific configurations | ||
| use rdkafka::consumer::{Consumer, StreamConsumer}; | ||
| use rdkafka::ClientConfig; | ||
|
|
||
| #[tokio::main] | ||
| async fn main() -> Result<(), Box<dyn std::error::Error>> { | ||
| // Configure an rdkafka consumer with DSH settings | ||
| let consumer: StreamConsumer = ClientConfig::new() | ||
| .set_dsh_consumer_config() | ||
| .create()?; | ||
|
|
||
| fn main() -> Result<(), Box<dyn std::error::Error>>{ | ||
| let dsh_properties = Properties::get(); | ||
| // get a rdkafka consumer config for example | ||
| let consumer: StreamConsumer = dsh_properties.consumer_rdkafka_config().create()?; | ||
| // Your application logic here | ||
|
|
||
| Ok(()) | ||
| } | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Connect to DSH | ||
| The SDK is compatible with running in a container on a DSH tenant, on DSH System Space, on a machine with Kafka Proxy/VPN or on a local machine to a local Kafka. | ||
| See [CONNECT_PROXY_VPN_LOCAL](CONNECT_PROXY_VPN_LOCAL.md) for more info. | ||
|
|
||
| ## Feature flags | ||
| The following features are available in this library and can be enabled/disabled in your Cargo.toml file.: | ||
| This SDK accommodates multiple deployment environments: | ||
| - Running in a container on a DSH tenant | ||
| - Running in DSH System Space | ||
| - Running on a machine with Kafka Proxy/VPN | ||
| - Running locally with a local Kafka instance | ||
|
|
||
| For more information, see the [CONNECT_PROXY_VPN_LOCAL.md](CONNECT_PROXY_VPN_LOCAL.md) document. | ||
|
|
||
| | **feature** | **default** | **Description** | | ||
| |---|---|---| | ||
| | `bootstrap` | ✓ | Generate signed certificate and fetch datastreams info <br> Also makes certificates available, to be used as lowlevel API | | ||
| | `rest-token-fetcher` | ✗ | Fetch tokens to use DSH Rest API | | ||
| | `mqtt-token-fetcher` | ✗ | Fetch tokens to use DSH MQTT | | ||
| | `metrics` | ✓ | Enable (custom) metrics for your service | | ||
| | `graceful_shutdown` | ✓ | Create a signal handler for implementing a graceful shutdown | | ||
| | `dlq` | ✗ | Dead Letter Queue implementation (experimental) | | ||
| | `rdkafka-ssl` | ✓ | Dynamically link to librdkafka to a locally installed OpenSSL | | ||
| | `rdkafka-ssl-vendored` | ✗ | Build OpenSSL during compile and statically link librdkafka <br> (No initial install required in environment, slower compile time) | | ||
| --- | ||
|
|
||
| See api documentation for more information on how to use these features including. | ||
| ## Feature Flags | ||
|
|
||
| ## Environment variables | ||
| The default RDKafka config can be overwritten by setting environment variables. See [ENV_VARIABLES.md](ENV_VARIABLES.md) for more information. | ||
| > **Important** | ||
| > The feature flags have changed since the `v0.5.X` update. Check the [migration guide](https://github.com/kpn-dsh/dsh-sdk-platform-rs/wiki/Migration-guide-(v0.4.X-%E2%80%90--v0.5.X)) for details. | ||
| Below is an overview of the available features: | ||
|
|
||
| ## Api doc | ||
| See the [api documentation](https://docs.rs/dsh_sdk/latest/dsh_sdk/) for more information on how to use this library. | ||
| | **Feature** | **Default?** | **Description** | **Example** | | ||
| |------------------------------|--------------|--------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------| | ||
| | `bootstrap` | ✓ | Certificate signing process and datastreams property retrieval | [Kafka](./examples/kafka_example.rs) / [Kafka Proxy](./examples/kafka_proxy.rs) | | ||
| | `kafka` | ✓ | Enables `DshKafkaConfig` trait and config structs for DSH Kafka connections | [Kafka](./examples/kafka_example.rs) / [Kafka Proxy](./examples/kafka_proxy.rs) | | ||
| | `rdkafka-config` | ✓ | Provides a `DshKafkaConfig` implementation for RDKafka | [Kafka](./examples/kafka_example.rs) / [Kafka Proxy](./examples/kafka_proxy.rs) | | ||
| | `schema-store` | ✗ | Interacts with the DSH Schema Store | [Schema Store API](./examples/schema_store_api.rs) | | ||
| | `protocol-token-fetcher` | ✗ | Fetches tokens for DSH MQTT and HTTP protocol adapters | [Token fetcher](./examples/protocol_token_fetcher.rs) / [Specific claims](./examples/protocol_token_fetcher_specific_claims.rs) | | ||
| | `management-api-token-fetcher` | ✗ | Fetches tokens for the DSH Management API | [Token fetcher](./examples/management_api_token_fetcher.rs) | | ||
| | `metrics` | ✗ | Adds Prometheus metrics (via an HTTP server) and re-exports `metrics` crate | [Expose metrics](./examples/expose_metrics.rs) / [Custom metrics](./examples/custom_metrics.rs) | | ||
| | `graceful-shutdown` | ✗ | Adds a Tokio-based graceful shutdown handler | [Graceful shutdown](./examples/graceful_shutdown.rs) | | ||
| | `dlq` | ✗ | Dead Letter Queue implementation | [Full example](./examples/dlq_implementation.rs) | | ||
|
|
||
| ### Selecting Features | ||
|
|
||
| To pick only the features you need, disable the default features and enable specific ones. For instance, if you only want the Management API Token Fetcher: | ||
|
|
||
| ```toml | ||
| [dependencies] | ||
| dsh_sdk = { version = "0.5", default-features = false, features = ["management-api-token-fetcher"] } | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Environment Variables | ||
|
|
||
| This SDK uses certain environment variables to configure connections to DSH. For a full list of supported variables and their usage, see [ENV_VARIABLES.md](ENV_VARIABLES.md). | ||
|
|
||
| --- | ||
|
|
||
| ## Examples | ||
| See folder [dsh_sdk/examples](/examples/) for simple examples on how to use the SDK. | ||
|
|
||
| ### Full service example | ||
| See folder [example_dsh_service](../example_dsh_service/) for a full service, including how to build the Rust project and post it to Harbor. See [readme](../example_dsh_service/README.md) for more information. | ||
| You can find simple usage examples in the [`examples/` directory](./examples/). | ||
|
|
||
| ### Full Service Example | ||
|
|
||
| A more complete example is provided in the [`example_dsh_service/`](../example_dsh_service/) directory, showcasing: | ||
|
|
||
| - How to build the Rust project | ||
| - How to package and push it to Harbor | ||
| - An end-to-end setup of a DSH service | ||
|
|
||
| See the [README](../example_dsh_service/README.md) in that directory for more information. | ||
|
|
||
| --- | ||
|
|
||
| ## Changelog | ||
| See [CHANGELOG.md](CHANGELOG.md) for all changes per version. | ||
|
|
||
| All changes per version are documented in [CHANGELOG.md](CHANGELOG.md). | ||
|
|
||
| --- | ||
|
|
||
| ## Contributing | ||
| See [CONTRIBUTING.md](../CONTRIBUTING.md) for more information on how to contribute to this project. | ||
|
|
||
| Contributions are welcome! For details on how to help improve this project, please see [CONTRIBUTING.md](../CONTRIBUTING.md). | ||
|
|
||
| --- | ||
|
|
||
| ## License | ||
| See [LICENSE](../LICENSE) for more information on the license for this project. | ||
|
|
||
| This project is licensed under the [Apache License 2.0](../LICENSE). | ||
|
|
||
| --- | ||
|
|
||
| ## Security | ||
| See [SECURITY.md](../SECURITY.md) for more information on the security policy for this project. | ||
|
|
||
| For information about the security policy of this project, including how to report vulnerabilities, see [SECURITY.md](../SECURITY.md). | ||
|
|
||
| --- | ||
| _Copyright (c) Koninklijke KPN N.V._ | ||
|
|
||
| © Koninklijke KPN N.V. | ||
|
|