Here are the steps involved in setting up YARN + MR2 using Cloudera Manager. There are 2 different processing engines that can be configured using Cloudera Manager (Map Reduce which is a legacy framework and YARN + MR2).
We don’t need to worry much about the legacy framework.
- Choose the drop-down of cluster Cluster 1
- Click on Add Service
- Choose YARN
- Configure Resource Manager – bigdataserver-4
- Configure Node Managers – bigdataserver-5 to bigdataserver-7 (same as Datanodes)
- Once the YARN is started
- Check whether bigdataserver-1 is configured as Gateway
- If not click on the instances and choose Add Role Instances
- Configure bigdataserver-1 and all other nodes as gateway nodes for YARN.
- Gateways facilitate us to connect to YARN cluster and issue commands to process data using frameworks like Map Reduce, Spark etc.
- Let us also review a few things as part of the setup process.
https://gist.github.com/dgadiraju/36759b0e00eedd86f1458843c53047ec