From 04c811e9856a8a5d8f292ea1bb56316202178a7b Mon Sep 17 00:00:00 2001 From: Prashasti Agarwal <144427971+prashastia@users.noreply.github.com> Date: Sun, 26 Jan 2025 12:26:39 +0530 Subject: [PATCH] Add snippet to enable Flink Metrics to `README.md` (#201) /gcbrun --- README.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/README.md b/README.md index 8291b192..16276dc8 100644 --- a/README.md +++ b/README.md @@ -540,6 +540,20 @@ supports collection and reporting of the following metrics in BigQuery sink: | `numberOfRecordsWrittenToBigQuerySinceCheckpoint` | Counter to keep track of the number of records successfully written to BigQuery since the last checkpoint. | At-least Once Sink | | `numberOfRecordsBufferedByBigQuerySinceCheckpoint` | Counter to keep track of the number of records currently buffered by the Storage Write API stream before committing them to the BigQuery Table. These records will be added to the Table following [Two Phase Commit Protocol's](https://nightlies.apache.org/flink/flink-docs-release-1.20/api/java/org/apache/flink/api/connector/sink2/Committer.html) `commit()` invocation. | Exactly Once Sink | +### Viewing Flink Metrics +* Flink offers a variety of metric reporters which the users could use to view these metrics. +* Flink’s [Metric Reporters](https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/) defines various pre-supported reporters that could be used to visualize metrics. + * A basic example would be logging in the Flink Log File using [slf4J reporter](https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/#slf4j). + * The following config needs to be added to `flink-conf.yaml` to enable reporting to the log file: +```yaml +// Enabling reporting and setting the reporter to slf4j +metrics.reporter.slf4j.class: org.apache.flink.metrics.slf4j.Slf4jReporter +// Fine Tune the Reporter Configuration +metrics.reporter.slf4j.interval: