tip: 369
title: Node support prometheus metrics
author: halibobo1205@gmail.com
discussions to: https://github.com/tronprotocol/TIPs/issues/369
status: Final
type: Standards Track
category: Core
created: 2022-03-05
Monitor your Java-tron server with Prometheus and Grafana.
Use prometheus, we can monitor node status, block information, transactions, api calls, method calls, mining block status, etc, then use grafana to show infos, make alerts, and be able to better maintain node running.
Currently, the popular blockchain basically supports open-source monitoring prometheus, java-tron has its own monitoring api, but it is not compatible with prometheus.If you want to use the current monitoring, you need to build your own front-end visualization. It is very unfriendly for users to monitor their own nodes.
Prometheus is an open-source systems monitoring and alerting toolkit .Prometheus collects and stores its metrics as time series data, i.e. metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels.
In layperson terms, metrics are numeric measurements, time series mean that changes are recorded over time. What users want to measure differs from application to application. For a web server it might be request times, for a database it might be number of active connections or number of active queries etc.
Metrics play an important role in understanding why your application is working in a certain way. Let's assume you are running a web application and find that the application is slow. You will need some information to find out what is happening with your application. For example the application can become slow when the number of requests are high. If you have the request count metric you can spot the reason and increase the number of servers to handle the load.
Expose the mertic API using simpleclient and simpleclient_httpserver. Please visit client_java#readme for more details.
Add two new settings to the config.conf file as follows.
net {
type = mainnet
# type = testnet
}
... ...
node {
... ...
p2p {
version = 11111 # 11111: mainnet; 20180622: testnet
}
####### add for prometheus start.
metrics{
prometheus{
enable=true
port="9527"
}
}
####### add for prometheus end.
}
| Name | Description |
|---|---|
| tron:header_status | Number , timestamp and hash of the most recent block. |
| tron::header_height | Number of the most recent block. |
| tron:header_time | Timestamp of the most recent block. |
| tron:http_service_latency_seconds | Latency for http api each url. |
| tron:grpc_service_latency_seconds | Latency for grpc api each endpoint . |
| tron:miner_latency_seconds | Latency from SR produce block to other node push block for every SR. |
| tron:http_bytes | Network traffic for http api each url. |
| tron:tcp_bytes | Network traffic for node TCP. |
| tron:udp_bytes | Network traffic for node UDP. |
| tron:manager_queue_size | The number of transactions in pending. |
| tron:txs | The number of transactions for node has executed through pushBlock,or bad transactions . |
| tron:block_fork | Number of fork. |
| tron:p2p_error | Number of P2P error. |
| tron:p2p_disconnect | Number of P2P disconnect. |
| tron:miner_latency | Estimated number of highest block. |
| tron:miner | Number of mined block for every SR. |
| tron:peers | Number of peers. |
| tron:db_size_bytes | DB size for every db. |
| tron:db_sst_level | DB sst level file distribution . |
this is use Hotspot collectors api
| Export Name | Description |
|---|---|
| StandardExports | Exports the standard exports common across all prometheus clients.This includes stats like CPU time spent and memory usage. |
| MemoryPoolsExports | Exports metrics about JVM memory areas. |
| MemoryAllocationExports | Total bytes allocated in a given JVM memory pool. Only updated after GC, not continuously. |
| BufferPoolsExports | Exports metrics about JVM buffers. |
| GarbageCollectorExports | Exports metrics about JVM garbage collectors. |
| ThreadExports | Exports metrics about JVM thread areas. |
| ClassLoadingExports | Exports metrics about JVM classloading. |
| VersionInfoExports | Exports JVM version info. |
| Name | Description |
|---|---|
| system_available_cpus | the number of processors available to the java virtual machine. |
| system_cpu_load | the system cpu load. |
| system_load_average | the system load average for the last minute. |
| system_total_physical_memory_bytes | total physical memory bytes. |
| system_free_physical_memory_bytes | free physical memory. |
| system_total_swap_spaces_bytes | total swap spaces. |
| system_free_swap_spaces_bytes | free swap spaces. |
We provide the standard prometheus http API, please visit prometheus for more detail.
Using open source monitoring prometheus, and data dashboards grafana,monitor and expose metrics with prometheus, use simpleclient to record metrics, simpleclient_httpserver to expose api. Query, visualize, alert on, and understand metrics with grafana.
TODO