Skip to content

Commit

Permalink
Add snippet to enable Flink Metrics to README.md (#201)
Browse files Browse the repository at this point in the history
/gcbrun
  • Loading branch information
prashastia authored Jan 26, 2025
1 parent 917c2c0 commit 04c811e
Showing 1 changed file with 14 additions and 0 deletions.
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -540,6 +540,20 @@ supports collection and reporting of the following metrics in BigQuery sink:
| `numberOfRecordsWrittenToBigQuerySinceCheckpoint` | Counter to keep track of the number of records successfully written to BigQuery since the last checkpoint. | At-least Once Sink |
| `numberOfRecordsBufferedByBigQuerySinceCheckpoint` | Counter to keep track of the number of records currently buffered by the Storage Write API stream before committing them to the BigQuery Table. These records will be added to the Table following [Two Phase Commit Protocol's](https://nightlies.apache.org/flink/flink-docs-release-1.20/api/java/org/apache/flink/api/connector/sink2/Committer.html) `commit()` invocation. | Exactly Once Sink |

### Viewing Flink Metrics
* Flink offers a variety of metric reporters which the users could use to view these metrics.
* Flink’s [Metric Reporters](https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/) defines various pre-supported reporters that could be used to visualize metrics.
* A basic example would be logging in the Flink Log File using [slf4J reporter](https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/#slf4j).
* The following config needs to be added to `flink-conf.yaml` to enable reporting to the log file:
```yaml
// Enabling reporting and setting the reporter to slf4j
metrics.reporter.slf4j.class: org.apache.flink.metrics.slf4j.Slf4jReporter
// Fine Tune the Reporter Configuration
metrics.reporter.slf4j.interval: <TIME INTERVAL> //e.g. 10 SECONDS
```
* Once the config is modified to enable reporting metrics:
* View the metric values in your application's log file.
* These could also be viewed using the Flink Metrics UI **during runtime**.
## Example Application
Expand Down

0 comments on commit 04c811e

Please sign in to comment.