Pulsar集群搭建部署-sqtce-51CTO博客
一、集群组成说明1. 搭建 Pulsar 集群至少需要 3 个组件：ZooKeeper 集群、BookKeeper 集群和 broker 集群（Broker 是 Pulsar 的自身实例)。这三个集群组件如下：ZooKeeper 集群（3 个 ZooKeeper 节点组成）bookie 集群（也称为 BookKeeper 集群，3 个 BookKeeper 节点组成）broker 集群（3 个 Pulsar 节点组成）2. Pulsar 的安装包已包含了搭建集群所需的各个组件库。无需单独下载 ZooKeeper 安装包和 BookKeeper 安装包。
二、安装JDK
在 3 台 Linux 裸机服务器上安装 JDK（要求版本不低于 JDK 8）安装略过
下载最新版本pulsar,https://pulsar.apache.org/download/?上传至3台服务器
解压安装包到指定目录
$tar -zxvf apache-pulsar-2.3.2-bin.tar.gz -C /data
三、配置部署 ZooKeeper 集群
/data/apache-pulsar-2.3.2$?cd?/data/apache-pulsar-2.3.2/conf
/data/apache-pulsar-2.3.2$?vim zookeeper.conf
# The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial? # synchronization phase can take initLimit=10 # The number of ticks that can pass between? # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just? # example sakes. dataDir=/data/zookeeper/zookeeper dataLogDir=/data/zookeeper/logs # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients maxClientCnxns=300 # # Be sure to read the maintenance section of the? # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir autopurge.snapRetainCount=180 # Purge task interval in hours # Set to "0" to disable auto purge feature autopurge.purgeInterval=180 server.1=HOSTNAME:2888:3888 server.2=HOSTNAME:2888:3888 server.3=HOSTNAME:2888:3888 |
# 新建文件目录/data/apache-pulsar-2.3.2$?mkdir -PV /data/zookeeper/zookeeper
/data/apache-pulsar-2.3.2$?mkdir -PV?/data/zookeeper/logs
# 新建文件 myid，分别在指定的sever上写入配置文件中指定的编号server.1服务器上
echo 1 > data/zookeeper/zookeeper/myid
server.2服务器上
/data/apache-pulsar-2.3.2$?echo 2 > data/zookeeper/zookeeper/myid
server.3服务器上
/data/apache-pulsar-2.3.2$?echo 3 > data/zookeeper/zookeeper/myid
# 执行后台运行命令/data/apache-pulsar-2.3.2$?bin/pulsar-daemon start zookeeper
# 执行 zookeeper 客户端连接命令/data/apache-pulsar-2.3.2$?bin/pulsar zookeeper-shell
#注意Enter 键进入命令行界面后，可完全使用 ZooKeeper 的各种命令，如ls、get等命令。使用quit命令退出
#在任一个 ZooKeeper 节点，初始化集群元数据。
/data/apache-pulsar-2.3.2$bin/pulsar initialize-cluster-metadata \
? --cluster pulsar-cluster-zk \
? --zookeeper HOSTNAMEIP:2181 \
? --configuration-store HOSTNAMEIP:2181 \
? --web-service-url http://HOSTNAMEIP:8080 \
? --web-service-url-tls https://HOSTNAMEIP:8443 \
? --broker-service-url pulsar://HOSTNAMEIP:6650 \
? --broker-service-url-tls pulsar+ssl://HOSTNAMEIP:6651
#查看集群元数据是否初始化成功，看到如下图所内容，表示初始化成功
$ bin/pulsar zookeeper-shell
四、配置部署 BookKeeper 集群
编辑 bookkeeper.conf 文件
/data/apache-pulsar-2.3.2$?vim conf/bookkeeper.conf?
advertisedAddress=HOSTNAMEIP zkServers=ZKSERVER1:2181,ZKSERVER2:2181,ZKSERVER3:2181 journalDirectory=/data/bookkeeper/journal ledgerDirectories=/data/bookkeeper/ledgers |
#创建bookie所需要目录
/data/apache-pulsar-2.3.2$?mkdir -pv?/data/bookkeeper/
#执行初始化元数据命令；若出现提示，输入 Y，继续（只需在一个bookie节点执行一次）/data/apache-pulsar-2.3.2$?bin/bookkeeper shell metaformat
#以后台进程启动bookie/data/apache-pulsar-2.3.2$?bin/pulsar-daemon start bookie#按照以上步骤，启动另外两个 bookie 节点。
如果出现报错
就是/data/bookkeeper/journal/current/VERSION? ?/data/bookkeeper/ledgers/current/VERSION?这两个文件与zk上的文件不一致造成的，将journal?和ledgers文件删除，在把zk的注册信息删除，重新注册一遍就好
验证是否启动成功/data/apache-pulsar-2.3.2$?bin/bookkeeper shell bookiesanity
/data/apache-pulsar-2.3.2$??vim logs/pulsar-bookie-HOSTNAME.out.2?
如果验证出现以上错误。
#解决方案：查看8000端口是否被占用
$egrep -v '^#|^$' conf/bookkeeper.conf
将prometheusStatsHttpPort=8000?端口更改掉
将bookie关闭
/data/apache-pulsar-2.3.2$?$bin/pulsar-daemon stop bookie
启动bookie
/data/apache-pulsar-2.3.2$?$bin/pulsar-daemon start bookie
验证是否启动成功，出现如下显示，表示启动成功Bookie
/data/apache-pulsar-2.3.2$?bin/bookkeeper shell bookiesanity
五、部署配置 Broker 集群
/data/apache-pulsar-2.3.2$ vim conf/broker.conf?
zookeeperServers=ZKSERVER1:2181,ZKSERVER2:2181,ZKSERVER3:2181 configurationStoreServers=ZKSERVER1:2181,ZKSERVER2:2181,ZKSERVER3:2181 advertisedAddress=HOSTNAME clusterName=pulsar-cluster-zk |
#以后台进程启动 broker/data/apache-pulsar-2.3.2$?bin/pulsar-daemon start broker
查看集群 brokers 节点情况/data/apache-pulsar-2.3.2$?bin/pulsar-admin brokers list pulsar-cluster
六、docker 安装 apachepulsar/pulsar-dashboard
#docker安装略过
#docker run --name pulsar-dashboard -dit -p 80:80 -e SERVICE_URL=http://PULSARSEVERIP:8080? apachepulsar/pulsar-dashboard
通过浏览器访问docker?宿主机IP
- 文章
- 推荐
- 热门新闻