快速完成Prometheus搭建
架構概述：
Prometheus架構
其大概的工作流程是：
Prometheus server 定期從配置好的 jobs 或者 exporters 中拉 metrics，或者接收來自 Pushgateway 發過來的 metrics，或者從其他的 Prometheus server 中拉 metrics。Prometheus server 在本地儲存收集到的 metrics，並執行已定義好的 alert.rules，記錄新的時間序列或者向 Alertmanager 推送警報。Alertmanager 根據配置檔案，對接收到的警報進行處理，發出告警。在圖形介面中，視覺化採集資料。軟體名 描述 訪問地址
prometheus 用於儲存監控資料，時序資料庫 http://192.168.10.10:9090/
node-exporter 用於機器資訊收集，安裝在目標機器，將監控資料打到api上（不同的exporter，會有不同的埠） http://192.168.10.10:9100/metrics
alertmanager 用於監控閥值傳送告警 http://192.168.10.10:9093
grafana 使用者Prometheus資料前端展示 http://192.168.10.10:3000 admin admin
-----------------------------------------------------------------------------------------------
1.安裝Prometheus編寫指令碼prometheus.sh ,內容如下
#!/bin/bashPROMETHEUS_VERSION="2.23.0"wget https://github.com/prometheus/prometheus/releases/download/v${PROMETHEUS_VERSION}/prometheus-${PROMETHEUS_VERSION}.linux-amd64.tar.gztar -xzvf prometheus-${PROMETHEUS_VERSION}.linux-amd64.tar.gzcd prometheus-${PROMETHEUS_VERSION}.linux-amd64/# if you just want to start prometheus as root#./prometheus --config.file=prometheus.yml# create useruseradd --no-create-home --shell /bin/false prometheus # create directoriesmkdir -p /etc/prometheusmkdir -p /var/lib/prometheus# set ownershipchown prometheus:prometheus /etc/prometheuschown prometheus:prometheus /var/lib/prometheus# copy binariescp prometheus /usr/local/bin/cp promtool /usr/local/bin/chown prometheus:prometheus /usr/local/bin/prometheuschown prometheus:prometheus /usr/local/bin/promtool# copy configcp -r consoles /etc/prometheuscp -r console_libraries /etc/prometheuscp prometheus.yml /etc/prometheus/prometheus.ymlchown -R prometheus:prometheus /etc/prometheus/consoleschown -R prometheus:prometheus /etc/prometheus/console_libraries# setup systemdecho '[Unit]Description=PrometheusWants=network-online.targetAfter=network-online.target[Service]User=prometheusGroup=prometheusType=simpleExecStart=/usr/local/bin/prometheus \ --config.file /etc/prometheus/prometheus.yml \ --storage.tsdb.path /var/lib/prometheus/ \ --web.console.templates=/etc/prometheus/consoles \ --web.console.libraries=/etc/prometheus/console_libraries[Install]WantedBy=multi-user.target' > /etc/systemd/system/prometheus.servicesystemctl daemon-reloadsystemctl enable prometheussystemctl start prometheus執行指令碼,完成Prometheus的安裝
chmod +x prometheus.shsh prometheus.sh可以訪問Prometheus的UI http://192.168.10.10:9090
-----------------------------------------------------------------------------------------------
2.安裝node-exporter編寫指令碼node_exporter.sh
#!/bin/bashNODE_EXPORTER_VERSION="1.0.1"wget https://github.com/prometheus/node_exporter/releases/download/v${NODE_EXPORTER_VERSION}/node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gztar -xzvf node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64.tar.gzcd node_exporter-${NODE_EXPORTER_VERSION}.linux-amd64cp node_exporter /usr/local/bin# create useruseradd --no-create-home --shell /bin/false node_exporterchown node_exporter:node_exporter /usr/local/bin/node_exporterecho '[Unit]Description=Node ExporterWants=network-online.targetAfter=network-online.target[Service]User=node_exporterGroup=node_exporterType=simpleExecStart=/usr/local/bin/node_exporter[Install]WantedBy=multi-user.target' > /etc/systemd/system/node_exporter.service# enable node_exporter in systemctlsystemctl daemon-reloadsystemctl start node_exportersystemctl enable node_exporter#下面這段需要在prometheus.yml中新增echo "Setup complete.Add the following lines to /etc/prometheus/prometheus.yml: - job_name: 'node_exporter' scrape_interval: 5s static_configs: - targets: ['localhost:9100']"執行指令碼,完成node_exporter的安裝
chmod +x node_exporter.shsh node_exporter.sh在去修改一下Prometheus的配置檔案/etc/prometheus/prometheus.yml
- job_name: 'node_exporter' scrape_interval: 5s static_configs: - targets: ['localhost:9100']重啟Prometheus,讓配置生效
systemctl restart prometheus可以訪問Prometheus的UI檢視是否監控到了Node_exporter資料http://192.168.10.10:9090/targets
-----------------------------------------------------------------------------------------------
3.安裝grafana執行下面的命令，透過yum方式安裝grafana
# officer site: https://grafana.com/docs/grafana/latest/installation/rpm/#install-manually-with-yum# localhost:3000# install grafana repositoryecho '[grafana]name=grafanabaseurl=https://packages.grafana.com/oss/rpmrepo_gpgcheck=1enabled=1gpgcheck=1gpgkey=https://packages.grafana.com/gpg.keysslverify=1sslcacert=/etc/pki/tls/certs/ca-bundle.crt' > /etc/yum.repos.d/grafana_oss.reporpm --import https://packages.grafana.com/gpg.key#install grafanayum install grafana -ysystemctl daemon-reloadsystemctl enable grafana-serversystemctl start grafana-server 安裝完成，訪問grafana地址
http://192.168.10.10:3000
預設使用者名稱: admin
預設密碼: admin
grafana登陸介面
-----------------------------------------------------------------------------------------------
4.安裝alertmanager編寫指令碼alertmanager.sh
#!/bin/bashALERTMANAGER_VERSION="0.21.0"wget https://github.com/prometheus/alertmanager/releases/download/v${ALERTMANAGER_VERSION}/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gztar xvzf alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gzcd alertmanager-${ALERTMANAGER_VERSION}.linux-amd64/# if you just want to start prometheus as root#./alertmanager --config.file=simple.yml# create useruseradd --no-create-home --shell /bin/false alertmanager # create directoriesmkdir /etc/alertmanagermkdir /etc/alertmanager/templatemkdir -p /var/lib/alertmanager/data# touch config filetouch /etc/alertmanager/alertmanager.yml# set ownershipchown -R alertmanager:alertmanager /etc/alertmanagerchown -R alertmanager:alertmanager /var/lib/alertmanager# copy binariescp alertmanager /usr/local/bin/cp amtool /usr/local/bin/# set ownershipchown alertmanager:alertmanager /usr/local/bin/alertmanagerchown alertmanager:alertmanager /usr/local/bin/amtool# setup systemdecho '[Unit]Description=Prometheus Alertmanager ServiceWants=network-online.targetAfter=network.target[Service]User=alertmanagerGroup=alertmanagerType=simpleExecStart=/usr/local/bin/alertmanager \ --config.file /etc/alertmanager/alertmanager.yml \ --storage.path /var/lib/alertmanager/dataRestart=always[Install]WantedBy=multi-user.target' > /etc/systemd/system/alertmanager.servicesystemctl daemon-reloadsystemctl enable alertmanagersystemctl start alertmanager# restart prometheussystemctl restart prometheusecho "(1/2)Setup complete.Add the following lines and substitute with correct values to /etc/alertmanager/alertmanager.yml:global: smtp_smarthost: 'localhost:25' smtp_from: '[email protected]' smtp_auth_username: '' smtp_auth_password: '' smtp_require_tls: falsetemplates:- '/etc/alertmanager/template/*.tmpl'route: repeat_interval: 1h receiver: operations-teamreceivers:- name: 'operations-team' email_configs: - to: '[email protected]' slack_configs: - api_url: https://hooks.slack.com/services/XXXXXX/XXXXXX/XXXXXX channel: '#prometheus-course' send_resolved: true " echo "(2/2)Setup complete.Alter the following config in /etc/prometheus/prometheus.yml:alerting: alertmanagers: - static_configs: - targets: - localhost:9093" 執行指令碼,完成alertmanager的安裝
chmod +x alertmanager.shsh alertmanager.sh修改alertmanager配置檔案/etc/alertmanager/alertmanager.yml，配置告警方式Email 還是 Slack
global: smtp_smarthost: 'localhost:25' smtp_from: '[email protected]' smtp_auth_username: '' smtp_auth_password: '' smtp_require_tls: falsetemplates:- '/etc/alertmanager/template/*.tmpl'route: repeat_interval: 1h receiver: operations-teamreceivers:- name: 'operations-team' email_configs: - to: '[email protected]' slack_configs: - api_url: https://hooks.slack.com/services/XXXXXX/XXXXXX/XXXXXX channel: '#prometheus-course' send_resolved: true修改Prometheus配置檔案/etc/prometheus/prometheus.yml ，關聯Alertmanager和Prometheus
alerting: alertmanagers: - static_configs: - targets: - localhost:9093" 重啟Prometheus
systemctl restart prometheus訪問alertmanager頁面 http://192.168.10.10:9093
alertmanager介面
至此，我們已經完成了Prometheus環境的搭建。
node_exporter 收集節點資訊
Prometheus 抓取node資料
granfana 前端展示
alertmanager 告警通知