Priority Aware Queue Management and its Relationship with Quota and Accounting

Abstract

If a user sends a large number of jobs simultaneously as in, for example, High Energy Physics analysis at CERN, it is called a job burst. If such a burst of jobs by a single user or multiple users arrives at a site, then it becomes impossible for the scheduler to serve them without using some queuing and job export mechanism. In the absence of this queue or export mechanism, it is likely that some of the jobs will be lost by the scheduler. Therefore, a decentralized and self-organizing scheduling system is required which can not only automatically export jobs to its peers under severe load conditions, but it can also manage its own scheduling policies, hierarchy, queue and network conditions. The required queuing mechanism at each peer should follow some management scheme which can associate priorities with each job inside the queue, depending on the user profile and job requirements, and the scheduler should service high priority jobs preferentially thus ensuring better Grid service standards.