根据grafana+prometheus组件来监控中间件，例如kafka集群，mysql数据库，rabbitmq集群，mongodb集群和redis集群。整个类似于埋点的功能，收集的metrics参数送给grafana图形化界面展示。在建立这些exporter时，需要注意本机器的端口使用情况和集群角色的建立，如rabbitmq它有监控着的角色。grafana选择有很多，建议选择需要界面展示重点参数的面板。

监控kafka集群：

下载地址：wget https://github.com/danielqsj/kafka_exporter/releases/download/v1.2.0/kafka_exporter-1.2.0.linux-amd64.tar.gz

在prometheus.yml文件中加入：  

- job_name: 'kafka'

     static_configs:

      - targets: ['kafkaIP或者域名:9308']

        labels:

          instance: kafka@kafkaIP或者域名

例如：        

- job_name: 'kafka'

  static_configs:

    - targets:

      - '192.168.xx.01:9308'

      - '192.168.xx.02:9308'

      - '192.168.xx.03:9308'

 

监控mysql数据库：      

由于mysqld_exporter需要连接到MySQL，创建用户并赋予所需的权限

mysql> GRANT REPLICATION CLIENT,PROCESS ON *.* TO 'mysql_monitor'@'localhost' identified by 'mysql_monitor';

mysql> GRANT SELECT ON *.* TO 'mysql_monitor'@'localhost';

 

 

创建.my.cnf配置文件

$ vim /usr/local/mysqld_exporter-0.10.0.linux-amd64/.my.cnf

[client]

user=mysql_monitor

password=mysql_monitor

 

执行脚本：nohup /usr/local/mysqld_exporter-0.10.0.linux-amd64/mysqld_exporter --config.my-cnf="/usr/local/mysqld_exporter-0.10.0.linux-amd64/.my.cnf" &

 

 

监控rabbitmq集群：

 

https://github.com/kbudde/rabbitmq_exporter/releases

tar zxvf rabbitmq_exporter-0.20.0.linux-amd64.tar.gz

 

cd   rabbitmq_exporter-0.20.0.linux-amd64.tar.gz

 

启动  RABBIT_USER=aisino RABBIT_PASSWORD=hubeiyfzx2019 OUTPUT_FORMAT=JSON RABBIT_URL=http://192.168.xx.xx:15672 nohup ./rabbitmq_exporter >rabbitmq.log 2>&1 &

 

 

注意RabbitMQ的用户必须具有monitoring权限，否则会报如下错误

 

{"error":"Error while retrieving data from rabbitHost","level":"warning","msg":"retrieving node failed","time":"2019-09-19T20:40:49+08:00"}

{"error":null,"host":"http://192.168.10.24:15672","level":"error","msg":"Error while retrieving data from rabbitHost","statusCode":401,"time":"2019-09-19T20:40:49+08:00"}

{"error":"Error while retrieving data from rabbitHost","level":"warning","msg":"retrieving queue failed","time":"2019-09-19T20:40:49+08:00"}

 

 

RABBIT_USER=guest RABBIT_PASSWORD=guest OUTPUT_FORMAT=JSON PUBLISH_PORT=9099 RABBIT_URL=http://192.168.2.4:15672 nohup  ./rabbitmq_exporter >/dev/null 2>&1 &

 

 

Promthues.yml 添加监控

  - job_name：'RabbitMq'

    static_configs：

      - 目标：['localhost：9099']

 

grafana监控：

 在控制面板加入4371，导入UI模板

 

监控MongoDB集群

https://github.com/percona/mongodb_exporter      

      1、mongodb的插件安装：

wget https://github.com/dcu/mongodb_exporter/releases/download/v1.0.0/mongodb_exporter-linux-amd64

tar -zxvf mongodb_exporter-linux-amd64 -C /usr/local/prometheus

cd /usr/local/prometheus

mongod配置副本集启动

nohup ./mongodb_exporter -mongodb.uri mongodb://192.168.10.xx:27017,192.168.10.xx:27017,192.168.10.xx:27017 > mongodb.log 2>&1 &

mongodb配置单机启动：

/mongodb_exporter -mongodb.uri mongodb://192.168.152.128:27017

 

监控Redis集群     

目前监控redis集群有一个痛点，无法通过一个固定的ip:port来访问集群，现在redis的exporter，需要监控每个节点的redis，从而达到监控整个redis集群。

 nohup ./redis_exporter -web.listen-address=0.0.0.0:9121 -redis.addr redis://192.168.10.24:6380,redis://192.168.10.24:6381 -redis.password aisino &

  - job_name: 'redis_exporter_targets'

    static_configs:

      - targets:

        - redis://first-redis-host:6379

        - redis://second-redis-host:6379

        - redis://third-redis-host:6380

 

        nohup ./redis_exporter -web.listen-address=0.0.0.0:9121 -redis.addr redis://192.168.10.xx:6380 -redis.password aisino &

        nohup ./redis_exporter -web.listen-address=0.0.0.0:9122 -redis.addr redis://192.168.10.xx:6380 -redis.password aisino > 6380.log 2>&1 &

        nohup ./redis_exporter -web.listen-address=0.0.0.0:9121 -redis.addr redis://192.168.10.xx:6379 -redis.password aisino > 6379.log 2>&1 &