Let us see the details with respect to commissioning or decommissioning a node. Decommissioning is nothing but stopping roles on the cluster.
As discussed earlier, we have 3 types of Nodes. Behavior is a bit different for each type of Node.
- Gateway
- Master
- Maintenance – Decommission -> Complete Maintenance -> Recommission.
- Removing the hosts – Decommission -> Remove from the cluster. We need to migrate the master components of services to other masters in the cluster and then delete the components before decommissioning the node. It is better to work on one master at a time and it is not very common.
- Typically when we decommission masters, we end up decommissioning the entire cluster.
- Worker (we will see a demo on bigdataserver-8).
- Maintenance – Multiple servers at a time. Decommission -> Complete Maintenance -> Recommission. When it comes to HDFS, we have options either to Decommission or Take Offline. For short term maintenance, we should use the Take Offline option.
- Removing the hosts – Decommission -> Remove from the cluster. Don’t use Take Offline option while removing from the cluster.
- When we decommission worker nodes, they will be temporarily discarded by the masters.
- Namenode will ignore those nodes to copy any data.
- Resource Manager will not use those nodes for processing the data.
- Same is the case with Impala, HBase etc.
Decommissioning a Node from Cluster
Note: We also have the option to Decommission Datanode or Take Offline for worker nodes on top of Decommission Hosts.
We cannot decommission a Datanode or a node with a Datanode if the number of Datanodes equals the replication factor (which by default is three).
Steps for decommission
- Go to Hosts
- Select the host or hosts that you want to decommission
- Click on Action -> Select Hosts Decommission
- Another pop-up window displaying the services that are running on that node
- Click Confirm to decommission on that particular node
Recommission
Applicable only for hosts decommissioned using Cloudera Manager.
Steps for Recommission
- Click on the Hosts tab.
- Select one or more decommissioned hosts to recommission
- Select Actions for Selected -> Hosts Recommission
- A Recommission Command pop-up window displays for Confirmation
- Once you confirm pop-up window shows each step in the recommission command as it is run.
- Once recommission is successful, Select Actions for Selected -> Click on Start Roles on Hosts to start all roles related to that particular node.
Note: If host/node and roles are marked as commissioned, they need not be started.