Configure Capacity Scheduler

Let us see how we can configure Capacity Scheduler using Cloudera Manager.

  • Log in to Cloudera Manager and go to YARN and then click on Configuaration
  • Search for “Scheduler”
  • And then define the property with multiple queues in the property “Capacity Scheduler Configuration Advanced Configuration Snippet (Safety Valve)” with different Name and Value as below.
    • yarn.scheduler.capacity.root.queues – prod,dev, and qa
    • yarn.scheduler.capacity.root.capacity – 100
    • yarn.scheduler.capacity.root.prod.capacity – 50
    • yarn.scheduler.capacity.root.dev.capacity – 10
    •  yarn.scheduler.capacity.root.qa.capacity – 40
  • And then Click on “Save Changes”.
  • And then set the property “yarn.resourcemanager.scheduler.class” to “org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.CapacityScheduler” to activate the Capacity Scheduler.
  • And then restart the yarn service to “Deploying the client configuration”.
  • Validate the scheduler configuration from the Resource Manager UI – <bigdataserver-3>:8088/cluster/scheduler

Share this post