Let us see how we can configure Capacity Scheduler using Cloudera Manager.
- Log in to Cloudera Manager and go to YARN and then click on Configuaration
- Search for “Scheduler”
- And then define the property with multiple queues in the property “Capacity Scheduler Configuration Advanced Configuration Snippet (Safety Valve)” with different Name and Value as below.
- yarn.scheduler.capacity.root.queues – prod,dev, and qa
- yarn.scheduler.capacity.root.capacity – 100
- yarn.scheduler.capacity.root.prod.capacity – 50
- yarn.scheduler.capacity.root.dev.capacity – 10
- yarn.scheduler.capacity.root.qa.capacity – 40
- And then Click on “Save Changes”.
- And then set the property “yarn.resourcemanager.scheduler.class” to “org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.CapacityScheduler” to activate the Capacity Scheduler.
- And then restart the yarn service to “Deploying the client configuration”.
- Validate the scheduler configuration from the Resource Manager UI – <bigdataserver-3>:8088/cluster/scheduler