As part of this section we will see how to set up HDFS components such as Namenode, Secondary Namenode, Datanodes etc while exploring some of the key concepts of this very important service. Setup HDFS Copy Data into HDFS Components of HDFS Configuration Files and Important Properties Review Web UIs and log files Checkpointing and …
Zookeeper have bunch of commands to get the information about the services managed by Zookeeper as well as Zookeeper itself. Make sure to create softlink for java executable so that zkCli.sh can run. zkCli.sh or zookeeper-shell.sh – It is primarily to get details about services Here are some sample commands. Be careful while running rmr …
Quite often we will have distributed services running on multiple servers (especially masters, for e.g.: Active/Passive Namenode of HDFS). Zookeeper can keep track of the multiple servers associated with a distributed service and ensure that it serves the clients of that service with High Availability and Fault Tolerance. Zookeeper service handles the following functions in …
Let us review some of the important properties associated with Zookeeper. Use Cloudera Manager and go to configs Also go to /etc/zookeeper/conf to review the properties dataDir dataLogDir – for snapshots of transaction logs. tickTime – used for heartbeat clientPort server.* Cloudera Manager manages run time properties in unique way. When ever you restart a …
This is the first CDH service that we are going to configure on the cluster. It is open source application which manages distributed applications in simple and reliable manner. Once setup is completed one of the servers will be elected as the leader and all other servers are followers. Followers receive message proposals from the …
We need to follow the standard process while setting up any software. Downloading and Installing – already taken care as part of adding hosts to the cluster. Architecture and Configuration – we need to understand architecture and plan for the configuration. Architecture – Zookeeper have all master architecture. Other services (typically masters of those services) …
