This is the first CDH service that we are going to configure on the cluster.
- It is open source application which manages distributed applications in simple and reliable manner.
- Once setup is completed one of the servers will be elected as the leader and all other servers are followers. Followers receive message proposals from the leader and agree upon message delivery to the clients.
- It is recommended to install zookeeper server on three or more nodes (using odd number).
- In larger clusters we might have 5 or 7 Zookeeper instances
- Zookeeper cluster is also called as Zookeeper Ensemble. Quorum (2 out of 3 in 3 node cluster) is number of nodes which determine commit. In 5 node cluster, quorum will constitute 3 nodes and in 7 node cluster it is 4.
- We will be using Zookeeper for below services when we configure them as part of the cluster.
- HDFS Namenode HA
- Resource Manager HA
- HBase Masters configuration
- Kafka Brokers
- and more
Following are the steps associated with setting up of Zookeeper.
- Select hosts on which zookeeper to be installed. We will be setting up on bigdataserver-2, bigdataserver-3 and bigdataserver-4. If you are using non GCP platform make sure to choose those servers which are categorized as Masters.
- Select Data and Transaction log directory
- /var/lib/zookeeper is assigned for both Data and Transaction logs. However, they can be customized. We will be using sub directory zookeeper under /data1. /dev/sdb which is additional disk attached to each of the servers is formatted and mounted on directory /data1.
- Install and start zookeeper service
Validation and Troubleshooting
Once setup is done it is very important to validate the service and also to understand how to troubleshoot in case of any issues.
- Go to dataDir and check the files such as myId as well as snapshots of the services registered with Zookeeper.
- Go to /var/log/zookeeper on all the hosts where service is configured and check relevant files to make sure that there are no issues related the services configured and started.