How Prometheus works
By periodically capturing the status of monitored components through HTTP, any component can be connected to Prometheus monitoring as long as it provides the corresponding HTTP interface and conforms to the data format defined by Prometheus. Prometheus Server is responsible for fetching metrics (metrics) data on targets on a regular basis, and each target needs to expose an HTTP service interface for Prometheus fetching. This method of calling the monitored object to obtain monitoring data is called Pull. The Pull approach reflects Prometheus's unique design philosophy, which is different from most monitoring using the Push approach.
Advantage
- A powerful multidimensional data model
- Flexible and powerful query statement (PromQL): In the same query statement, multiple metrics can be multiplied, added, joined, scored bits and so on.
- Easy to manage: Prometheus server is a separate binary file that works directly locally and does not depend on distributed storage.
- Efficient: Only 3.5 bytes per sample point on average, and one Prometheus server can handle millions of metrics.
- Collecting time series data using pull mode not only facilitates native testing but also avoids bad metrics pushed by problematic servers.
- Time series data can be pushed to Prometheus server using push gateway
- Monitoring targets can be obtained through service discovery or static configuration.
- There are many visual graphical interfaces.
- Easy to scale.
Characteristic - Multidimensional data model: Time series data identified by measure name and key-value pairs.
- PromSQL: A flexible query language that can use multidimensional data to complete complex queries.
- Instead of relying on distributed storage, a single server node can work directly.
- Time series data is collected by pull based on HTTP.
- Push time series data is supported by the PushGateway component.
- Discover targets through service discovery or static configuration.
- grafana is supported in a variety of graphics modes and dashboards.
Framework
From this schema diagram, you can also see that the main modules of Prometheus include Server, Exporters, Pushgateway, PromQL, Alertmanager, WebUI, etc. It roughly uses the logic: - Prometheus server periodically pulls data from statically configured targets or from targets discovered by services.
- When the newly pulled data is larger than the configured memory cache, Prometheus persists the data to disk (if remote storage is used to persist to the cloud).
- Prometheus can configure rules, then periodically query data, and when a condition triggers, it pushes alert to the configured aletmanager.
- When Alertmanager receives a warning, it can configure, aggregate, de-weigh, de-noise, and finally send a warning.
- You can query and aggregate data using API s, Prometheus Console, or Grafana.
Deploy Prometheus
Download promethues
wget https://github.com/prometheus/prometheus/releases/download/v2.31.1/prometheus-2.31.1.linux-amd64.tar.gz
Unzip and rename
[root@master ~]# tar xf prometheus-2.31.1.linux-amd64.tar.gz -C /usr/local/ [root@master ~]# cd /usr/local/ [root@master local]# ls bin include libexec share etc lib prometheus-2.31.1.linux-amd64 src games lib64 sbin [root@master local]# mv prometheus-2.31.1.linux-amd64/ prometheus [root@master local]# ls bin games lib libexec sbin src etc include lib64 prometheus share [root@master local]# useradd -M -s /sbin/nologin prometheus [root@master local]# mkdir -p /data/prometheus [root@master local]# chown -R prometheus.prometheus /usr/local/prometheus /data/prometheus
Configure service file
[root@master ~]# vim /usr/lib/systemd/system/prometheus.service [Unit] Description=The Prometheus Server After=network.target [Service] Restart=on-failure ExecStart=/usr/local/prometheus/prometheus --config.file=/usr/local/prometheus/prometheus.yml RestartSec=15s [Install] WantedBy=multi-user.target [root@master ~]# systemctl daemon-reload
Start Services
[root@master ~]# systemctl restart prometheus [root@master ~]# ss -anlt State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 0.0.0.0:22 0.0.0.0:* LISTEN 0 128 *:9090 *:* LISTEN 0 128 [::]:22 [::]:*
Test Access