Monitoring
Metrics registration
Each Module has its own instance of "Statistics" class (m_statistics) which provides the interface for metrics monitoring. While defining your custom Module you can register the metric using the template function:
void registerMetric(T* pointer, std::string name, metrics::metric_type mtype)
in the configure()
function of your Module, where:
- pointer - pointer to the variable storing the metric value
- name - name of the metric published to database
- mtype - type of the metric
For example:
m_statistics->registerMetric<std::atomic<int> >(&m_metric6, "RandomMetric1-int", daqling::core::metrics::LAST_VALUE);
The acceptable types of variables are:
std::atomic<int>
std::atomic<float>
std::atomic<double>
std::atomic<bool>
std::atomic<size_t>
There are 4 possible metrics types:
LAST_VALUE
- measure the current value of metricsACCUMULATE
- accumulate the current variable value to metric value and reset the variableAVERAGE
- calculate the average value of metric over given time intervalRATE
- calculate the rate over given time interval
All of them are defined in the daqling::core::metrics
namespace.
Metrics configuration
The metrics values can be send directly to influxDB or published via ZMQ socket. The metrics destination can be set in the JSON configuration of the Module by specifying the following settings:
stats_uri
- URI of ZMQ publisher socketinfluxDb_uri
- URI used for sending the values directly to influxDBinfluxDb_name
- name of the influxDB database
Either stats_uri or influxDb_uri must be specified in order to allow for the metrics registration in the module. Example settings:
"settings": {
"stats_uri": "tcp://*:6007",
"influxDb_uri": "http://localhost:8086/write?db=",
"influxDb_name": "DAQlingMetrics"
}
In this configuration the values will be sent to the influxDB database as well as published via ZMQ publisher.
It is also possible to configure the interval between stats posting. This is done by adding the field:
* stats_interval
- interval between posting in ms. Default is 500.
Metrics visualization
The default metrics visualization tool is Grafana (https://grafana.com/) connected to influxDB (https://www.influxdata.com). Both Grafana and influxDB can be installed and configured automatically (see the installation instructions in README.md
file). In this case the influxDB database DAQlingMetrics is configured as well as default Grafana dashboard called DAQling demo.
The Grafana dashboard can be displayed under the following link: <host>:3000
- grafana default login: admin
- default password: admin
ZMQ subscriber
When "stats_uri"
option in the JSON Module settings is specified, the metrics values are published via ZMQ socket. Then it is possible to create a custom ZMQ subscriber. The example subscriber can be found here:
scripts/Monitoring/metric-manager.py
with a JSON configuration file: metrics-config.json
.
This subscriber gives the possibility to send the metrics values to Redis or/and InfluxDB. It is also possible to set the list of subscribed metrics as well as the individual destination of defined metric.
Metrics configuration:
"metrics": [
"<metric_name>,<destination>",
...
]
Possible destinations:
i
- InfluxDBr
- Redisb
- both (InfluxDB and Redis)
If the <destination>
parameter is not provided or not valid (e.g. w
), the i
option is set as default.
Example settings:
"metrics": [
"metricssimulator-RandomMetric1-int,i",
"metricssimulator-RandomMetric2-float,w",
"metricssimulator-RandomMetric3-double,b",
"metricssimulator-RandomMetric8-double_average,r"
]