First, let us run jobs without specifying any queue and see what happens.
Submit the Long running job to production queue by using the below command. This job requires 279 containers.
Submit the other job which requires only 18 containers to process the data.
Submitting jobs and Validating Capacity Scheduler
Let us submit few jobs and see how the resources are allocated using Capacity Scheduler.
- Submit the Long running job to the production queue.
- Submit the other jobs to the test and production queues.
- Submit another job to the qa queue
- Now the production queue has two apps. Since it is a Capacity Scheduler the two jobs will be executed once the resources are available in the production queue and the job in the test queue will be executed as it is with the required resources.
- Here is how the jobs will run with Capacity Scheduler