Introduction to Capacity Scheduler

Capacity Scheduler is nothing but FIFO Scheduler within each queue. Unlike FIFO Scheduler, Capacity Scheduler has multiple queues and users can submit jobs to a particular queue.

  • By default, Hortonworks Hadoop Distribution uses Capacity Scheduler.
  • Configuration files related to the capacity scheduler
    • yarn-site.xml
    • capacity-scheduler.xml
  • For setting up queues in Capacity Scheduler you need to make changes in the capacity-scheduler.xml configuration file.
  • Click here for the complete documentation of Capacity Scheduler (from Hortonworks).
  • E.g.: We will be defining the three primary queues as below.
    • prod – queue with 50% of total capacity
    • dev – queue with 10% of total capacity
    • qa – queue with 40% of total capacity

Share this post