We need to follow the standard process while setting up any software.
- Downloading and Installing – already taken care as part of adding hosts to the cluster.
- Architecture and Configuration – we need to understand architecture and plan for the configuration.
- Architecture – Zookeeper have all master architecture. Other services (typically masters of those services) in the cluster will act as clients. We will register services such as Namenode of HDFS, Resource Manager of YARN etc with Zookeeper later.
- Configuration Files – /etc/zookeeper/conf
- Daemon Processes – zookeeper
- Service logs – /var/log/zookeeper
- Service Data – based on the property dataDir
Cluster Topology
We are setting up cluster on 7+1 nodes. We start with 7 nodes and then we will add one more node later.
- Gateway(s) and Management Service
- bigdataserver-1
- Masters
- bigdataserver-2 – Zookeeper
- bigdataserver-3 – Zookeeper
- bigdataserver-4 – Zookeeper
- Slaves or Worker Nodes
- bigdataserver-5
- bigdataserver-6
- bigdataserver-7
- We will create host group zookeeper to run commands using ansible on all nodes where Zookeeper is running.