Introduction

As part of this section we will see how to enable HDFS Namenode High Availability as well as YARN Resource Manager High Availability while exploring key concepts.

• High Availability – Overview
• Configure HDFS Namenode HA
• Review Properties – HDFS Namenode HA
• HDFS Namenode HA – Key Concepts
• Configure YARN Resource Manager HA
• Review – YARN Resource Manager HA
• High Availability – Implications

Cluster Topology

We are setting up the cluster on 7+1 nodes. We start with 7 nodes and then we will add one more node later.

• Gateway(s) and Management Service
  • bigdataserver-1
• Masters
  • bigdataserver-2
    • Zookeeper
    • Active/Standby Namenode
  • bigdataserver-3
    • Zookeeper
    • Active/Standby Namenode
    • Active/Standby Resource Manager
  • bigdataserver-4
    • Zookeeper
    • Active/Standby Resource Manager
• Slaves or Worker Nodes
  • bigdataserver-5 – Datanode, Node Manager
  • bigdataserver-6 – Datanode, Node Manager
  • bigdataserver-7 – Datanode, Node Manager