【完全分布式Hadoop】(八)CentOS6.5搭建Zookeeper3.4.6集群

Hadoop admin 2年前 (2018-10-27) 284次浏览 0个评论 扫描二维码

由于后边搭建HDFS和Yarn高可用需要用到Zookeeper,所以这里现在集群中搭建Zookeeper

一. 分布式安装部署

1.0 下载地址

官网首页: https://zookeeper.apache.org/

下载地址: https://archive.apache.org/dist/zookeeper/
blob.jpg

1.1 集群规划

ZK配置规则:
HA 2n+1 奇数 做选举

生产上集群数量 ZK数量
50台规模以下 7台
50~100 9/11台
>100 11台

在hadoop000、hadoop001和hadoop002三个节点上部署Zookeeper,三个节点都已安装jdk。

IP 节点名称 Zookeeper JDK
192.168.142.150 hadoop000 Zookeeper JDK
192.168.142.151 hadoop001 Zookeeper JDK
192.168.142.152 hadoop002 Zookeeper JDK

hadoop用户在家目录下创建如下目录

[hadoop@hadoop000 ~]$ pwd
/home/hadoop
[hadoop@hadoop000 ~]$ ll
total 20
drwxrwxr-x. 2 hadoop hadoop 4096 Nov 27 22:23 app
drwxrwxr-x. 2 hadoop hadoop 4096 Nov 27 22:23 data
drwxrwxr-x. 2 hadoop hadoop 4096 Nov 27 22:23 lib
drwxrwxr-x. 2 hadoop hadoop 4096 Nov 27 22:23 software
drwxrwxr-x. 2 hadoop hadoop 4096 Nov 27 22:23 source

将zookeeper-3.4.6.tar.gz上传到/home/hadoop/software

[hadoop@hadoop000 software]$ ll
total 321576
-rw-r--r--. 1 hadoop hadoop 311585484 Nov 26 23:06 hadoop-2.6.0-cdh5.7.0.tar.gz
-rw-r--r--. 1 hadoop hadoop  17699306 Nov 26 23:03 zookeeper-3.4.6.tar.gz
[hadoop@hadoop000 software]$ pwd
/home/hadoop/software

1.2 设置hadoop用户的ssh免密

这里不做详细介绍,具体查看该系列文章前边的root免密的操作,注意:hadoop免密需要在hadoop用户下进行操作
【完全分布式Hadoop】(三)集群设置SSH无密码登陆

1.3 解压安装

(1)解压zookeeper安装包到/home/hadoop/app/目录下

[hadoop@hadoop000 software]$ tar -xzvf zookeeper-3.4.6.tar.gz -C /home/hadoop/app/

(2)设置ZOOKEEPER_HOME

vim ~/.bash_profile

export ZOOKEEPER_HOME=/home/hadoop/app/zookeeper-3.4.6
export PATH=$ZOOKEEPER_HOME/bin:$PATH

zookeeper home生效
source ~/.bash_profile

(3)在/home/hadoop/app/zookeeper-3.4.6/这个目录下创建data

[hadoop@hadoop000 zookeeper-3.4.6]$ mkdir data

(4)复制/home/hadoop/app/zookeeper-3.4.6/conf这个目录下的zoo_sample.cfg为zoo.cfg

[hadoop@hadoop000 conf]$ cp zoo_sample.cfg zoo.cfg

1.4 配置zoo.cfg文件

具体配置,修改dateDir,添加日志存放目录
[hadoop@hadoop000 conf]$ vim zoo.cfg

dataDir=/home/hadoop/app/zookeeper-3.4.6/data
dataLogDir=/home/hadoop/app/zookeeper-3.4.6/logs

末尾添加如下配置:

server.0=hadoop000:2888:3888
server.1=hadoop001:2888:3888
server.2=hadoop002:2888:3888

1.5 集群配置

(1)在/home/hadoop/app/zookeeper-3.4.6/data目录下创建一个myid的文件

[hadoop@hadoop000 data]$ touch myid

(2)编辑myid文件, 在文件中添加与server对应的编号:如

此处的配置参考:server.0=hadoop000:2888:3888
[hadoop@hadoop000 data]$ vim myid

0
(3)拷贝配置好的zookeeper到其他机器上
[hadoop@hadoop000 app]$ scp -r zookeeper-3.4.6/ hadoop@hadoop001:/home/hadoop/app/
[hadoop@hadoop000 app]$ scp -r zookeeper-3.4.6/ hadoop@hadoop002:/home/hadoop/app/

并分别修改hadoop001,hadoop002中myid文件中内容为1,2

[hadoop@hadoop001 data]$ echo 1 >myid
[hadoop@hadoop002 data]$ echo 2 >myid

检查myid是否写入成功

1.6 设置hadoop001和hadoop002的ZOOKEEPER_HOME

[hadoop@hadoop001 data]$ vim ~/.bash_profile
export ZOOKEEPER_HOME=/home/hadoop/app/zookeeper-3.4.6
export PATH=$ZOOKEEPER_HOME/bin:$PATH
[hadoop@hadoop001 data]$ source ~/.bash_profile
[hadoop@hadoop002 data]$ vim ~/.bash_profile
export ZOOKEEPER_HOME=/home/hadoop/app/zookeeper-3.4.6
export PATH=$ZOOKEEPER_HOME/bin:$PATH
[hadoop@hadoop002 data]$ source ~/.bash_profile

1.7 启动集群

(1)分别启动zookeeper
[hadoop@hadoop000 app]$ zkServer.sh start
JMX enabled by default
Using config: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[hadoop@hadoop001 app]$ zkServer.sh start
JMX enabled by default
Using config: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[hadoop@hadoop002 app]$ zkServer.sh start
JMX enabled by default
Using config: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
(2)查看状态
[hadoop@hadoop000 app]$ zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower
[hadoop@hadoop001 data]$ zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: leader
[hadoop@hadoop002 data]$ zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/app/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower
(3)停止zookeeper

一下只是示例,无需操作

[hadoop@hadoop000 app]$ zkServer.sh stop
[hadoop@hadoop001 app]$ zkServer.sh stop
[hadoop@hadoop002 app]$ zkServer.sh stop

codeobj , 版权所有丨如未注明 , 均为原创丨本网站采用BY-NC-SA协议进行授权
转载请注明原文链接:【完全分布式Hadoop】(八)CentOS6.5搭建Zookeeper3.4.6集群
喜欢 (0)
[a37free@163.com]
分享 (0)
发表我的评论
取消评论
表情 贴图 加粗 删除线 居中 斜体 签到

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址