Home Archives Categories Tags

RabbitMQ 集群配置

发布时间: 更新时间: 总字数:1131 阅读时间:3m 作者: 分享

RabbitMQ 集群配置。

准备

vim /etc/hosts

192.168.128.101    rabbitmq-101
192.168.128.102    rabbitmq-102
192.168.128.103    rabbitmq-103
systemctl stop firewalld.service

安装包

yum install rabbitmq-server

优化

systemctl enable rabbitmq-server
vim /usr/lib/systemd/system/rabbitmq-server.service
[Service]
LimitNOFILE = 300000
systemctl daemon-reload

修改 /etc/rabbitmq/rabbitmq.config:259/340

[
 {rabbit,
   ...
   {cluster_partition_handling, pause_minority}
   ...
  },
 {rabbitmq_management,
 ...
   {rates_mode, none} or
   {rates_mode, basic} or
   {rates_mode, detailed}
 ...
 }
]

启动集群

准备

第一个节点rabbitmq-101:

systemctl start rabbitmq-server
cp -rp /var/lib/rabbitmq/.erlang.cookie /tmp/.erlang.cookie

chown rabbitmq. /tmp/.erlang.cookie -R
chown rabbitmq.  /var/lib/rabbitmq/.erlang.cookie -R

chmod 0400 /var/lib/rabbitmq/.erlang.cookie

将上面的 .cookie 文件分发到其他节点

scp /var/lib/rabbitmq/.erlang.cookie rabbitmq-102:~
scp /var/lib/rabbitmq/.erlang.cookie rabbitmq-103:~

启动所有节点 RabbitMQ 服务:

systemctl start rabbitmq-server

组建集群

rabbitmqctl stop_app;
rabbitmqctl join_cluster rabbit@rabbitmq-101 --ram;
rabbitmqctl start_app;
[root@rabbitmq-102 ~]# rabbitmqctl stop_app;
Stopping node 'rabbit@rabbitmq-102' ...
[root@rabbitmq-102 ~]# rabbitmqctl join_cluster rabbit@rabbitmq-101 --ram;
Clustering node 'rabbit@rabbitmq-102' with 'rabbit@rabbitmq-101' ...
[root@rabbitmq-102 ~]# rabbitmqctl start_app
Starting node 'rabbit@rabbitmq-102' ...

设置 HA

rabbitmqctl set_policy ha-all '^(?!amq\.).*' '{"ha-mode": "all"}'

启动管理插件

节点执行:

rabbitmq-plugins enable rabbitmq_management
[root@rabbitmq-101 ~]# rabbitmq-plugins enable rabbitmq_management
The following plugins have been enabled:
  mochiweb
  webmachine
  rabbitmq_web_dispatch
  amqp_client
  rabbitmq_management_agent
  rabbitmq_management

Applying plugin configuration to rabbit@rabbitmq-101... started 6 plugins.
[root@rabbitmq-101 ~]# 

添加 monitor 用户

rabbitmqctl add_user monitor xiexianbin.cn
rabbitmqctl set_user_tags monitor monitoring
rabbitmqctl set_permissions -p / monitor ".*" ".*" ".*"

修改密码

rabbitmqctl change_password user_admin passwd_admin

设置 tag

rabbitmqctl set_user_tags user_admin administrator
# rabbitmqctl set_user_tags user_admin monitoring policymaker

设置权限

rabbitmqctl set_permissions [-p <vhost>] <user> <conf> <write> <read>
rabbitmqctl set_permissions -p / user_admin ".*" ".*" ".*"
rabbitmqctl set_permissions -p / user_admin ".*" "^$" ".*"
rabbitmqctl list_permissions -p /

查看集群状态

[root@rabbitmq-101 ~]# rabbitmqctl cluster_status
Cluster status of node 'rabbit@rabbitmq-101' ...
[{nodes,[{disc,['rabbit@rabbitmq-101']},{ram,['rabbit@rabbitmq-102']}]},
 {running_nodes,['rabbit@rabbitmq-102','rabbit@rabbitmq-101']},
 {cluster_name,<<"rabbit@rabbitmq-101">>},
 {partitions,[]},
 {alarms,[{'rabbit@rabbitmq-102',[]},{'rabbit@rabbitmq-101',[]}]}]
[root@rabbitmq-101 ~]# 

默认 master 节点为 disc 节点。

说明

内存使用

rabbitmq 内存占用量分析(针对bit-64位机器):

默认情况下,RabbitMQ 启动时内存限制为总内存量的 40%。可以在/etc/rabbitmq/rabbitmq.config配置:

   %% Resource Limits & Flow Control
   %% ==============================
   %%
   %% See http://www.rabbitmq.com/memory.html for full details.

   %% Memory-based Flow Control threshold.
   %%
   %% {vm_memory_high_watermark, 0.4},  # 配置占用比例

   %% Alternatively, we can set a limit (in bytes) of RAM used by the node.
   %%
   %% {vm_memory_high_watermark, {absolute, 1073741824}},  # 配置内存占用量,单位bytes
   %%
   %% Or you can set absolute value using memory units.
   %%
   %% {vm_memory_high_watermark, {absolute, "1024M"}},  # 配置内存占用量,指定单位M
   %%
   %% Supported units suffixes:
   %%
   %% k, kiB: kibibytes (2^10 bytes)
   %% M, MiB: mebibytes (2^20)
   %% G, GiB: gibibytes (2^30)
   %% kB: kilobytes (10^3)
   %% MB: megabytes (10^6)
   %% GB: gigabytes (10^9)

以上例说明:

机器物理内存:377G,启动RabbitMQ服务时,内存限制为总内存的40%,即150G左右,启动RabbitMQ时日志如下:

[root@xiexianbin_cn rabbitmq]# grep "Memory limit set" ./*
./rabbit@xiexianbin_cn.log-20170618:Memory limit set to 154706MB of 386766MB total.

进程占用内存说明:

[root@xiexianbin_cn ~]# cat /proc/28579/status 
Name:   beam.smp
State:  S (sleeping)
Tgid:   28579
Ngid:   0
Pid:    28579
PPid:   1
...
VmPeak: 17934548 kB  # 进程所占用最大虚拟内存大小
VmSize: 17883064 kB  # 进程当前虚拟内存大小
...
VmHWM:    506508 kB  # 进程所占用物理内存的峰值
VmRSS:    487876 kB  # 进程当前占用物理内存的大小(与procrank中的RSS)
VmData: 17715576 kB  # 进程数据段的大小
...

通过RabbitMQ查看内存占用量:

[root@xiexianbin_cn ~]# rabbitmqctl status
Status of node 'rabbit@xiexianbin_cn'
[{pid,28579},
...
 {memory,
     [{total,276131584},  # 实际使用内存,RabbitMQ manage 面板显示的内存使用大小
      {connection_readers,764504},
      {connection_writers,37440},
      {connection_channels,7569392},
      {connection_other,2074696},
      {queue_procs,2507696},
      {queue_slave_procs,57342936},
      {plugins,36284088},
      {other_proc,378664},
      {mnesia,808528},
      {metrics,304040},
      {mgmt_db,33125952},
      {msg_index,76312},
      {other_ets,2747784},
      {binary,48278352},
      {code,27598779},
      {atom,992409},
      {other_system,55541244}]},
...
 {vm_memory_high_watermark,0.4},  # 内存限比例
 {vm_memory_limit,162221613056},  # 内存限制量
 ...

常见问题

RabbitMQ 强制删除队列

有些情况下导致队列无法删除,可使用如下命令强制删除:

rabbitmqctl eval 'rabbit_amqqueue:internal_delete({resource,<<"/">>,queue,<<"cinder.info">>}).'

脑裂问题

现象是在 RabbitMQ GUI上显示

Network partition detected
Mnesia reports that this RabbitMQ cluster has experienced a network partition. There is a risk of losing data. Please read RabbitMQ documentation about network partitions and the possible solutions.

原因分析:

这是由于网络问题导致集群出现了脑裂

解决办法:

在相对不怎么信任的分区里,对那个分区的节点实行,在出现问题的节点上执行:

rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl start_app

加入集群失败问题

[root@rabbitmq-102 rabbitmq]# rabbitmqctl join_cluster rabbit@rabbitmq-101 --ram
Clustering node 'rabbit@rabbitmq-102' with 'rabbit@rabbitmq-101'
Error: {inconsistent_cluster,"Node 'rabbit@rabbitmq-101' thinks it's clustered with node 'rabbit@rabbitmq-102', but 'rabbit@rabbitmq-102' disagrees"}

现在在节点 rabbitmq-101 运行的节点执行:

rabbitmqctl forget_cluster_node rabbitmq-102
# 然后在 rabbitmq-102 执行
rabbitmqctl join_cluster rabbit@rabbitmq-101 --ram
In some cases the last node to go offline cannot be brought back up. It can be removed from the cluster using the forget_cluster_node rabbitmqctl command.

参考

最新评论
加载中...